System Architecture

​

Overview

​

Architecture Diagram

​

Core Components

​

Client (Frontend)

• Managing monitored websites
• Viewing real-time uptime status
• Analyzing historical metrics and trends
• Configuring status pages

• Server-side rendering for optimal performance
• tRPC client for type-safe API communication
• Real-time updates via WebSocket connections
• Responsive UI built with React and Tailwind CSS

​

Server (API Layer)

• Exposes type-safe RPC endpoints
• Handles authentication and authorization
• Manages WebSocket connections for real-time updates
• Orchestrates data access across PostgreSQL, ClickHouse, and Redis

• userRouter - User authentication and profile management
• websiteRouter - Website monitoring configuration
• statusPageRouter - Public status page management
• statusDomainRouter - Custom domain configuration

​

Publisher

• Queries PostgreSQL for active websites (every 3 minutes)
• Publishes website check tasks to Redis Streams
• Ensures only active websites are monitored
• Implements in-flight protection to prevent overlapping cycles

​

Worker

• Consume website check tasks from Redis Streams
• Execute HTTP requests to monitored URLs
• Record metrics (response time, status code, availability)
• Write results to ClickHouse for long-term storage
• Implement graceful error handling and PEL (Pending Entries List) management

• Consumer group support for horizontal scaling
• Automatic reclaim of stale messages
• Self-liveness monitoring and auto-recovery
• Timeout protection on all external operations

​

Component Interactions

​

Data Flow

​

1. Task Publishing

1. Query PostgreSQL for all active websites
2. Bulk publish to Redis Stream using XADD
3. Trim stream to prevent unbounded growth (~8000 messages max)
4. Wait for next interval (3 minutes)

​

2. Task Processing

1. Fresh messages: Read new tasks with XREADGROUP (blocking, 1 second)
2. Validation: Check if website is still active via Prisma
3. HTTP check: Execute request with configurable timeout (10 seconds)
4. Metrics recording: Batch insert to ClickHouse
5. Acknowledgment: Remove from Redis PEL with XACK
6. PEL reclaim: Automatically reclaim stale messages (idle > 5 minutes)

​

3. Data Retrieval

1. Configuration data: PostgreSQL (website settings, user data)
2. Metrics data: ClickHouse (time-series uptime events)
3. Real-time status: Redis (current processing state)
4. WebSocket push: Live updates to connected clients

​

Scalability

​

Horizontal Scaling

• Worker instances: Multiple workers can join the same Redis consumer group
• Region support: Workers can be deployed across different geographic regions
• Independent scaling: Each component (client, server, worker, publisher) scales independently

​

Data Partitioning

• ClickHouse: Ordered by (website_id, region_id, checked_at) for efficient queries
• Redis Streams: Consumer groups distribute work across workers
• PostgreSQL: Indexed for fast website lookups and user queries

​

Reliability

​

Fault Tolerance

• Redis reconnection: Automatic exponential backoff with jitter
• PEL management: Stale messages are automatically reclaimed
• Worker watchdog: Self-monitoring with automatic restart on freeze
• Timeout protection: All external operations have client-side timeouts

​

Data Consistency

• At-least-once delivery: Redis Streams with consumer groups
• Idempotent processing: Safe to process the same check multiple times
• ACK safety: Messages are ACKed even on processing failures to prevent PEL growth
• Publisher re-enqueue: Failed checks are retried on the next publishing cycle

​

Next Steps