Skip to main content

​
POST /api/analyze-image/claude

Analyze a handoff document or whiteboard image using Claude AI to extract patient information, vital signs, medications, tasks, and clinical notes.
Requires ANTHROPIC_API_KEY in environment variables.

​
Request

This endpoint accepts multipart/form-data for file uploads.
​
image
file
required
Image file of the handoff document or whiteboard. Supported formats: JPEG, PNG, WebP, GIF. Maximum file size: 10MB.

​
Request Example

curl -X POST http://localhost:3001/api/analyze-image/claude \
  -F "image=@/path/to/handoff-whiteboard.jpg"

​
Response

​
summary
string
AI-generated analysis of the handoff image, including:
  • Patient overview and Shift-Level Forecast
  • Current clinical status and vital signs
  • Key lab/imaging findings and trends
  • Active medications and treatments
  • Task status (completed and outstanding)
  • Safety concerns (allergies, fall risk, code status)
​
provider
string
AI provider used for analysis. Always returns "claude" for this endpoint.

​
Response Example

{
  "summary": "## Patient Overview\n\nPatient: John Doe, 67M\nMRN: 123456\nRoom: 401A\nDiagnosis: Acute MI\n\n## Current Clinical Status\n\nVitals (18:00):\n- BP: 142/88\n- HR: 78\n- SpO2: 96% on 2L NC\n- Temp: 98.6°F\n\nTrend: BP decreasing from 158/92 this morning (improved)\n\n## Active Medications\n\n- Aspirin 81mg daily\n- Metoprolol 25mg BID\n- Atorvastatin 40mg HS\n\n## Task Status\n\n### Completed Tasks\n✓ 14:00 - Troponin draw (HIGH priority)\n✓ 16:00 - EKG completed\n✓ 18:00 - Vitals check\n\n### Outstanding Tasks\n⚠ 20:00 - Evening medications (HIGH priority)\n⚠ 22:00 - Vitals recheck (MEDIUM priority)\n⚠ 00:00 - Repeat troponin (CRITICAL priority)\n\n## Shift-Level Forecast\n\n⛈️ High workload spike predicted at midnight\n- Troponin labs returning\n- Vitals rechecks\n- Possible MD notification if troponin elevated\n\n☀️ Stable period from 02:00-06:00\n\n## Safety Concerns\n\n- Allergies: Penicillin\n- Fall risk: LOW\n- Code status: Full Code",
  "provider": "claude"
}

​
Error Responses

​
Analysis Features

​
What Claude Extracts

The AI model (claude-sonnet-4-20250514) analyzes the image and extracts:

Patient Information

Name, age, MRN, room number, diagnosis

Vital Signs

BP, HR, SpO2, temperature, respiratory rate with trends

Clinical Data

Lab results, imaging findings, trends over time

Medications

Active medications, dosages, schedules

Tasks

Completed and outstanding tasks with priorities

Safety Info

Allergies, fall risk, code status, precautions

​
Shift-Level Forecast

The analysis includes a “weather forecast” for the shift:
  • High workload periods: Times when multiple tasks cluster
  • Stable periods: Expected quiet times
  • Predicted events: Anticipated lab results, scheduled procedures
  • Visual timeline: Color-coded workload intensity

​
Trend Highlighting

The AI emphasizes trends in:
  • Vital signs (improving, worsening, stable)
  • Lab values (rising troponin, normalizing electrolytes)
  • Clinical condition (deteriorating, stable, improving)

​
Best Practices

  • Use good lighting when photographing whiteboards
  • Ensure text is legible and in focus
  • Include the entire handoff document in frame
  • Avoid glare and reflections
  • JPEG/JPG: Best for photographs
  • PNG: Best for screenshots or digital documents
  • Keep files under 10MB
  • Combine with /api/summarize-record/claude for comprehensive handoffs
  • Pass the summary to the record summarization endpoint as imageAnalysis
  • This merges whiteboard info with electronic health records

​
Additional Notes

This application uses Claude Sonnet 4 exclusively. The implementation does not include OpenAI integration.

Build docs developers (and LLMs) love

Get started for freeTalk to us