Skip to main content

​
Overview

Workflows are the core abstraction in Chronoverse that define what should be executed and how often. A workflow represents a scheduled task that runs at specified intervals, with built-in failure tracking and automatic termination capabilities.

​
Workflow Types

Chronoverse supports two workflow types, each designed for different use cases:

​
HEARTBEAT

A simple health check workflow that verifies system availability.
  • Service health monitoring
  • Endpoint availability checks
  • System liveness probes
  • Network connectivity tests
  • Executes instantly with minimal overhead
  • Returns success/failure status
  • No payload configuration required
  • Ideal for high-frequency monitoring (every 1-5 minutes)
{
  "kind": "HEARTBEAT"
}

​
CONTAINER

Executes custom containerized applications and scripts in isolated Docker containers.
  • Data processing pipelines
  • ETL jobs
  • Backup and maintenance tasks
  • Custom business logic
  • Scheduled reports generation
  • Database cleanup operations
  • Pulls or builds Docker images
  • Executes in isolated containers
  • Captures stdout/stderr logs
  • Supports environment variables and volumes
  • Automatic cleanup after execution
{
  "kind": "CONTAINER",
  "image": "python:3.11-alpine",
  "command": ["python", "-c"],
  "args": ["print('Hello from Chronoverse!')"],
  "env": {
    "API_KEY": "your-api-key",
    "ENVIRONMENT": "production"
  }
}

​
Workflow Properties

Each workflow has the following properties:
​
id
string
required
Unique identifier (UUID) for the workflow
​
user_id
string
required
ID of the user who owns the workflow
​
name
string
required
Human-readable name for the workflow
​
kind
string
required
Workflow type: HEARTBEAT or CONTAINER
​
payload
string
required
JSON string containing workflow configuration (varies by kind)
​
interval
integer
required
Execution interval in minutes (minimum: 1)
​
max_consecutive_job_failures_allowed
integer
required
Maximum number of consecutive failures before auto-termination
​
consecutive_job_failures_count
integer
Current count of consecutive failures (read-only)
​
build_status
string
Current build status for CONTAINER workflows:
  • QUEUED: Waiting to be processed
  • STARTED: Build in progress
  • COMPLETED: Ready for execution
  • FAILED: Build failed
  • CANCELED: Build was canceled
​
created_at
timestamp
Workflow creation time (RFC3339 format)
​
updated_at
timestamp
Last update time (RFC3339 format)
​
terminated_at
timestamp
Termination time if workflow is terminated (RFC3339 format)

​
Workflow Lifecycle

​
1. Creation

When a workflow is created:
  1. User submits workflow definition via API
  2. System validates the payload and parameters
  3. Workflow is stored in PostgreSQL
  4. For CONTAINER workflows, build status is set to QUEUED
  5. Workflow Worker picks up CONTAINER workflows and prepares execution environment
HEARTBEAT workflows are immediately ready for execution and don’t require a build phase.

​
2. Building (CONTAINER only)

The Workflow Worker:
  • Validates the Docker image and configuration
  • Prepares execution templates
  • Stores configuration in Redis for fast access
  • Updates build status accordingly
Workflows with FAILED or CANCELED build status won’t be scheduled for execution.

​
3. Scheduling

The Scheduling Worker continuously:
  1. Polls PostgreSQL for workflows due for execution
  2. Calculates next execution time based on interval
  3. Creates job entries in the jobs table
  4. Publishes job events to Kafka’s workflows topic
next_execution = last_execution_time + (interval * 60 seconds)

if current_time >= next_execution:
  schedule_new_job()

​
4. Execution

When a job is scheduled:
  1. Workflow Worker (for CONTAINER) or Execution Worker (for HEARTBEAT) consumes the event
  2. Job status changes from PENDING → QUEUED → RUNNING
  3. Execution happens in isolated environment
  4. Logs are captured and sent to Kafka’s job_logs topic
  5. Job completes with COMPLETED or FAILED status

​
5. Failure Tracking

After each job execution:
  • On Success: consecutive_job_failures_count is reset to 0
  • On Failure: consecutive_job_failures_count is incremented
When consecutive_job_failures_count reaches max_consecutive_job_failures_allowed, the workflow is automatically terminated to prevent resource waste.

​
6. Termination

Workflows can be terminated:
  1. Manually: User terminates via API
  2. Automatically: After reaching max consecutive failures
  3. Via Deletion: Workflow deletion also terminates it
Terminated workflows:
  • Stop being scheduled for execution
  • Retain all historical data
  • Can be identified by non-null terminated_at timestamp
  • Cannot be reactivated (create a new workflow instead)

​
Build Status Details

For CONTAINER workflows, the build status indicates readiness:

QUEUED

Workflow is waiting in the build queue. The Workflow Worker will pick it up soon.

STARTED

Workflow Worker is currently building the execution environment. This includes validating the Docker image and preparing configuration.

COMPLETED

Build successful. Workflow is ready for execution and will be scheduled according to its interval.

FAILED

Build failed due to invalid configuration, missing image, or other errors. Check logs for details. The workflow won’t be scheduled until fixed.

CANCELED

Build was canceled before completion. This can happen if the workflow is updated or deleted during build.

​
Payload Configuration

​
HEARTBEAT Payload

Heartbeat workflows have minimal configuration: 
{
  "kind": "HEARTBEAT"
}

​
CONTAINER Payload

Container workflows support extensive configuration: 
{
  "kind": "CONTAINER",
  "image": "python:3.11-alpine",
  "command": ["python", "-c"],
  "args": [
    "import requests; response = requests.get('https://api.example.com/health'); print(f'Status: {response.status_code}')"
  ],
  "env": {
    "API_ENDPOINT": "https://api.example.com",
    "TIMEOUT": "30"
  },
  "working_dir": "/app"
}
​
image
string
required
Docker image to use (e.g., python:3.11-alpine, node:20-alpine, custom registry images)
​
command
array
Override the default command of the image (e.g., ["python", "-c"])
​
args
array
Arguments to pass to the command
​
env
object
Environment variables as key-value pairs
​
working_dir
string
Working directory inside the container

​
Best Practices

​
Interval Selection

  • Use HEARTBEAT workflows
  • Monitor critical services
  • Keep payloads simple
  • Set reasonable failure thresholds (e.g., 3-5)
  • Suitable for most CONTAINER workflows
  • Data processing, backups, reports
  • Balance between timeliness and resource usage
  • Set higher failure thresholds (e.g., 5-10)
  • Heavy processing tasks
  • Large data operations
  • Batch processing
  • Set appropriate timeouts

​
Failure Handling

1

Set Appropriate Thresholds

Consider the nature of your workflow:
  • Flaky external APIs: Higher threshold (5-10)
  • Critical internal services: Lower threshold (2-3)
  • Experimental workflows: Medium threshold (3-5)
2

Monitor Failure Counts

Regularly check consecutive_job_failures_count and investigate:
  • Approaching threshold? Address underlying issues
  • Auto-terminated? Fix and recreate workflow
3

Use Notifications

Enable real-time notifications to catch failures early

​
Container Optimization

  • Use Alpine Images: Smaller, faster to pull (e.g., python:3.11-alpine)
  • Minimize Layers: Keep Docker images lean
  • Cache Images: Frequently used images are cached by Docker
  • Set Resource Limits: Prevent resource exhaustion
  • Handle Signals: Implement graceful shutdown in your code

​
Next Steps

Create Workflow

Learn how to create your first workflow

Jobs

Understand job execution and monitoring

Workers

Learn how workers process workflows

API Reference

Complete workflow API documentation

Build docs developers (and LLMs) love

Get started for freeTalk to us