Hedge

Overview

The Hedge pattern (also known as “hedged requests” or “tail latency reduction”) sends multiple redundant requests to the same operation and returns the first successful response. This is useful for reducing tail latency when dealing with unpredictable response times.

When to Use

Reducing tail latency (P99, P99.9) in distributed systems
When you have multiple replicas that can serve the same request
Operations where consistency between requests is guaranteed
Systems where redundant work is acceptable for better latency

How It Works

Initial Request

Send the first request immediately.

Wait

Wait for a configured delay.

Hedge Request

If the first request hasn’t completed, send another request.

First Wins

Return the first successful response, cancel others.

API Reference

Function Signature

function withHedge<T extends AnyFn>(fn: T, options: HedgeOptions): T

Options

count

number

required

Total number of requests to send (including the initial request). Must be at least 2.

delay

number

required

Delay in milliseconds between sending each hedged request.

Examples

Basic Hedging

import { withHedge } from 'cruel'

const fetchUser = async (id: string) => {
  const response = await fetch(`https://api.example.com/users/${id}`)
  return response.json()
}

// Send up to 3 requests, 100ms apart
const hedgedFetch = withHedge(fetchUser, {
  count: 3,
  delay: 100,
})

// First successful response wins
const user = await hedgedFetch('123')

Database Query Hedging

// Query multiple read replicas
const hedgedQuery = withHedge(
  async (sql: string) => {
    const replica = selectRandomReplica()
    return replica.query(sql)
  },
  {
    count: 2,     // Query 2 replicas
    delay: 50,    // 50ms apart
  }
)

const results = await hedgedQuery('SELECT * FROM users WHERE id = 1')

Microservice Hedging

// Call the same microservice endpoint multiple times
const hedgedServiceCall = withHedge(
  async (data: RequestData) => {
    // Service has multiple instances behind load balancer
    return await serviceClient.request(data)
  },
  {
    count: 3,
    delay: 100,
  }
)

Search Query Hedging

// Search across multiple data centers
const hedgedSearch = withHedge(
  async (query: string) => {
    const datacenter = selectDatacenter()
    return datacenter.search(query)
  },
  {
    count: 2,
    delay: 200,  // Give first DC 200ms before hedging
  }
)

const results = await hedgedSearch('example query')

CDN Request Hedging

// Fetch from multiple CDN edge locations
const hedgedCDNFetch = withHedge(
  async (url: string) => {
    const edge = selectRandomEdge()
    return fetch(`${edge.url}${url}`)
  },
  {
    count: 2,
    delay: 100,
  }
)

const asset = await hedgedCDNFetch('/images/logo.png')

Combining with Other Patterns

Hedge + Timeout

import { withHedge, withTimeout } from 'cruel'

// Each hedged request has its own timeout
const resilientAPI = withHedge(
  withTimeout(apiCall, { ms: 5000 }),
  {
    count: 3,
    delay: 100,
  }
)

Hedge + Retry

import { withHedge, withRetry } from 'cruel'

// Retry each hedged request individually
const resilientAPI = withHedge(
  withRetry(apiCall, {
    attempts: 2,
    delay: 500,
  }),
  {
    count: 2,
    delay: 200,
  }
)

Hedge + Circuit Breaker

import { withHedge, createCircuitBreaker } from 'cruel'

// Circuit breaker applies to all hedged requests
const resilientAPI = withHedge(
  createCircuitBreaker(apiCall, {
    threshold: 5,
    timeout: 30000,
  }),
  {
    count: 2,
    delay: 100,
  }
)

With Compose

import { cruel } from 'cruel'

const resilientAPI = cruel.compose(apiCall, {
  hedge: {
    count: 3,
    delay: 100,
  },
  timeoutMs: 5000,
  retry: {
    attempts: 2,
    backoff: 'exponential',
  },
})

Advanced Examples

Hedging with Target Selection

interface Target {
  url: string
  latency: number  // Historical average latency
}

function createSmartHedge(
  targets: Target[],
  hedgeCount: number = 2
) {
  return withHedge(
    async (endpoint: string) => {
      // Select target with lowest historical latency
      const sorted = [...targets].sort((a, b) => a.latency - b.latency)
      const target = sorted[Math.floor(Math.random() * hedgeCount)]
      
      const start = Date.now()
      const response = await fetch(`${target.url}${endpoint}`)
      
      // Update latency stats
      target.latency = (target.latency * 0.9) + ((Date.now() - start) * 0.1)
      
      return response.json()
    },
    {
      count: hedgeCount,
      delay: 50,
    }
  )
}

const smartAPI = createSmartHedge([
  { url: 'https://us-east.api.example.com', latency: 100 },
  { url: 'https://us-west.api.example.com', latency: 150 },
  { url: 'https://eu.api.example.com', latency: 200 },
], 2)

Adaptive Hedging

class AdaptiveHedge {
  private p95Latency: number = 100
  private successRate: number = 1

  createHedge<T extends AnyFn>(fn: T) {
    // Adjust hedging based on observed performance
    const shouldHedge = this.successRate < 0.95 || this.p95Latency > 200
    
    if (!shouldHedge) {
      return fn  // Don't hedge if performance is good
    }

    const hedgeDelay = Math.max(50, this.p95Latency * 0.5)
    const hedgeCount = this.successRate < 0.8 ? 3 : 2

    return withHedge(fn, {
      count: hedgeCount,
      delay: hedgeDelay,
    })
  }

  recordLatency(latency: number) {
    this.p95Latency = (this.p95Latency * 0.95) + (latency * 0.05)
  }

  recordSuccess(success: boolean) {
    this.successRate = (this.successRate * 0.95) + (success ? 0.05 : 0)
  }
}

const adaptive = new AdaptiveHedge()
const apiCall = adaptive.createHedge(fetchData)

Hedging with Cancellation

import { withHedge } from 'cruel'

function createCancelableHedge<T extends AnyFn>(
  fn: T,
  hedgeOptions: HedgeOptions
) {
  const controllers: AbortController[] = []

  const wrapped = withHedge(
    async (...args: Parameters<T>) => {
      const controller = new AbortController()
      controllers.push(controller)

      try {
        return await fn(...args, { signal: controller.signal })
      } finally {
        const idx = controllers.indexOf(controller)
        if (idx > -1) controllers.splice(idx, 1)
      }
    },
    hedgeOptions
  )

  // Cancel all in-flight requests
  const cancelAll = () => {
    controllers.forEach(c => c.abort())
    controllers.length = 0
  }

  return Object.assign(wrapped, { cancelAll })
}

const hedgedAPI = createCancelableHedge(apiCall, {
  count: 3,
  delay: 100,
})

// Later...
hedgedAPI.cancelAll()

Hedging with Cost Tracking

interface HedgeMetrics {
  totalRequests: number
  hedgedRequests: number
  wastedRequests: number
  latencyImprovement: number
}

function createTrackedHedge<T extends AnyFn>(
  fn: T,
  options: HedgeOptions
): T & { getMetrics: () => HedgeMetrics } {
  const metrics: HedgeMetrics = {
    totalRequests: 0,
    hedgedRequests: 0,
    wastedRequests: 0,
    latencyImprovement: 0,
  }

  const wrapped = async (...args: Parameters<T>): Promise<ReturnType<T>> => {
    metrics.totalRequests++
    let completedCount = 0
    let firstResponseTime = 0

    const tracked = async () => {
      const start = Date.now()
      try {
        const result = await fn(...args)
        const duration = Date.now() - start
        
        completedCount++
        if (firstResponseTime === 0) {
          firstResponseTime = duration
        }
        
        return result
      } catch (error) {
        completedCount++
        throw error
      }
    }

    try {
      const result = await withHedge(tracked, options)()
      
      const wasted = options.count - 1
      metrics.hedgedRequests += options.count
      metrics.wastedRequests += wasted
      
      return result
    } catch (error) {
      throw error
    }
  }

  return Object.assign(wrapped as T, {
    getMetrics: () => ({ ...metrics }),
  })
}

const trackedAPI = createTrackedHedge(apiCall, {
  count: 3,
  delay: 100,
})

setInterval(() => {
  const metrics = trackedAPI.getMetrics()
  console.log('Hedge metrics:', metrics)
}, 60000)

Error Handling

If all hedged requests fail, the error from the first request is thrown:

import { CruelError } from 'cruel'

try {
  await hedgedAPI()
} catch (error) {
  if (error.code === 'CRUEL_HEDGE_FAILED') {
    console.log('All hedged requests failed')
  }
}

Best Practices

Use for read operations: Hedging should only be used for idempotent operations
Don’t hedge writes: Avoid hedging for operations that modify state
Choose appropriate delays: Too short wastes resources, too long doesn’t help latency
Monitor wasted work: Track how many hedged requests are wasted
Consider costs: Hedging increases load and costs (compute, network, API calls)
Use with timeouts: Prevent slow requests from holding resources
Tune hedge count: More hedges improve latency but increase cost
Combine with retries carefully: Hedging + retries can create many requests

When NOT to Use Hedging

Write operations: Can cause duplicate writes
Non-idempotent operations: May have side effects
Rate-limited APIs: Wastes quota
Expensive operations: High compute/cost impact
Low latency variance: If P50 ≈ P99, hedging won’t help
Resource-constrained systems: Increases load

Configuration Guidelines

Scenario	Count	Delay	Reasoning
Database reads	2	50-100ms	Balance latency vs load
Search queries	2-3	100-200ms	High tail latency
CDN requests	2	50-100ms	Geographic variance
Microservices	2	100-200ms	Instance variance
Cache lookups	2	10-50ms	Very low latency

Performance Considerations

Latency Improvement

Hedging is most effective when:

P99 latency >> P50 latency (high tail latency)
Variance is high
Cost of extra work < cost of slow responses

Resource Cost

With hedge count N:

Best case: 1 request completes
Worst case: N requests complete
Average: ~1.5-2 requests complete

Example Metrics

// Without hedging
P50: 100ms, P99: 2000ms

// With hedging (count: 2, delay: 100ms)
P50: 100ms, P99: 500ms  // 75% improvement in P99
Cost: 1.5x requests on average

Get Started

Core Concepts

Chaos Types

Resilience Patterns

AI SDK Integration

Advanced

CLI

Overview

When to Use

How It Works

API Reference

Function Signature

Options

Examples

Basic Hedging

Database Query Hedging

Microservice Hedging

Search Query Hedging

CDN Request Hedging

Combining with Other Patterns

Hedge + Timeout

Hedge + Retry

Hedge + Circuit Breaker

With Compose

Advanced Examples

Hedging with Target Selection

Adaptive Hedging

Hedging with Cancellation

Hedging with Cost Tracking

Error Handling

Best Practices

When NOT to Use Hedging

Configuration Guidelines

Performance Considerations

Latency Improvement

Resource Cost

Example Metrics

Build docs developers (and LLMs) love

Get Started

Core Concepts

Chaos Types

Resilience Patterns

AI SDK Integration

Advanced

CLI

Documentation Index

​Overview

​When to Use

​How It Works

​API Reference

​Function Signature

​Options

​Examples

​Basic Hedging

​Database Query Hedging

​Microservice Hedging

​Search Query Hedging

​CDN Request Hedging

​Combining with Other Patterns

​Hedge + Timeout

​Hedge + Retry

​Hedge + Circuit Breaker

​With Compose

​Advanced Examples

​Hedging with Target Selection

​Adaptive Hedging

​Hedging with Cancellation

​Hedging with Cost Tracking

​Error Handling

​Best Practices

​When NOT to Use Hedging

​Configuration Guidelines

​Performance Considerations

​Latency Improvement

​Resource Cost

​Example Metrics

Build docs developers (and LLMs) love

Overview

When to Use

How It Works

API Reference

Function Signature

Options

Examples

Basic Hedging

Database Query Hedging

Microservice Hedging

Search Query Hedging

CDN Request Hedging

Combining with Other Patterns

Hedge + Timeout

Hedge + Retry

Hedge + Circuit Breaker

With Compose

Advanced Examples

Hedging with Target Selection

Adaptive Hedging

Hedging with Cancellation

Hedging with Cost Tracking

Error Handling

Best Practices

When NOT to Use Hedging

Configuration Guidelines

Performance Considerations

Latency Improvement

Resource Cost

Example Metrics