Admission Control and Weighted Fairness API Reference

ThrottleKit’s admission module provides higher-order building blocks for controlling whether work should be attempted at all, upstream of per-key rate limits. These APIs compose with the core Limiter interface via combineDecisions and cover adaptive client-side throttling, weighted max-min fairness, streaming token budgets, online learned reservations, analytics decoration, approximate sketch-based limiting, and multi-dimensional enforcement. All of these are exported from the top-level throttlekit package.

unifiedAdmission

Compose three orthogonal admission axes — rate, concurrency, and cost — into one UnifiedAdmitter via the combineDecisions algebra. Evaluation order is concurrency → rate → cost, first denial short-circuits.

import { unifiedAdmission } from "throttlekit";

function unifiedAdmission(options: UnifiedAdmissionOptions): UnifiedAdmitter

rate

Limiter

The rate axis — a Limiter returning a Decision for (key, 1) per admit call.

concurrency

ConcurrencyGuard

The concurrency axis — a ConcurrencyGuard from adaptiveConcurrency(). State is local (in-process); no store round trip.

cost

Limiter

The cost axis — a Limiter returning a Decision for (key, cost) where cost is the per-request weight.

backend

"sequential" | "lua-fused"

"sequential" (default) runs axes in order; first deny short-circuits. "lua-fused" collapses rate + cost into one Redis EVALSHA — requires the fused option group.

policy

"marginal" | "joint-lp"

"marginal" (default) admits when every axis independently allows. "joint-lp" additionally applies a bid-price filter (value ≥ p_R + p_C · cost). @experimental.

clock

Clock

Injectable time source. Defaults to the system clock.

const admitter = unifiedAdmission({
  rate: rateLimit({ strategy: gcra({ limit: 1000, periodMs: 60_000 }), store: redisStore }),
  cost: rateLimit({ strategy: tokenBucket({ limit: 100_000, ratePerSec: 1_000 }), store: redisStore }),
  concurrency: adaptiveConcurrency({ initialLimit: 20 }).guard,
});

const { decision, release } = await admitter.admit({ key: "user:42", cost: 50 });
try {
  if (!decision.allowed) return tooManyRequests(decision);
  const result = await doWork();
  return result;
} finally {
  release();
}

UnifiedAdmitter Methods

admit

(opts?: UnifiedAdmitOptions) => Promise<UnifiedAdmission>

Async admit. Works for any backend mix.

Show UnifiedAdmitOptions

key

string

Key passed to rate/cost axes. Default "" (the global bucket).

cost

number

Cost weight passed to the cost axis. Default 1.

value

number

Per-request value for the joint-LP bid-price test. Ignored unless policy: "joint-lp". Default 1.

hold

number

Expected service time for the 3-axis joint-LP concurrency term. Default 0.

admitSync

(opts?: UnifiedAdmitOptions) => UnifiedAdmission

Synchronous admit. Throws when any configured axis lacks a synchronous code path. Not available with backend: "lua-fused".

lastDecisions

() => Readonly<Partial<Record<UnifiedAxis, Decision | undefined>>>

Snapshot of the most recent admit’s per-axis decisions. Each call returns a fresh frozen object safe to leak into telemetry. Short-circuited axes are undefined.

UnifiedAdmission Fields

decision

Decision

The combined Decision across all configured axes.

release

(opts?: { dropped?: boolean }) => void

Release the held concurrency slot when work finishes. Pass dropped: true to signal overload. Idempotent; a no-op on denied admissions.

bindingAxis

UnifiedAxis | undefined

The axis whose denial bound this admission ("rate" / "concurrency" / "cost"), or undefined when admitted or when the joint-LP policy filter denied.

policyDenied

boolean

true when the admission was denied specifically by the joint-LP bid-price filter — every per-axis budget had slack, but value < p_R + p_C · cost. Absent or falsy under policy: "marginal" or any axis-bound denial.

tokenBudget

Windowed token-budget meter for post-hoc costs (e.g. LLM output tokens billed as they stream).

import { tokenBudget } from "throttlekit";

function tokenBudget(options: TokenBudgetOptions): TokenBudgetMeter

budget

number

required

Token budget L enforced per window. Floored to an integer; must be ≥ 1.

windowMs

number

required

Window width in ms. Windows are epoch-aligned.

clock

Clock

Injected clock. Defaults to the system clock.

const meter = tokenBudget({ budget: 100_000, windowMs: 60_000 });

for await (const tok of completion) {
  if (!meter.debitSync(1).allowed) break; // budget spent — stop generating
  emit(tok);
}

debitSync

(tokens?: number) => Decision

Atomically debit tokens (default 1) against the current window. Stop-at-boundary: a debit is admitted iff budget remains before it (served < L).

debit

(tokens?: number) => Promise<Decision>

Promise-returning form of debitSync. Resolves synchronously.

remaining

() => number

Tokens remaining in the current window (≥ 0). Rolls the window but does not debit.

reset

() => void

Forget all usage; the next call starts a fresh window.

distributedTokenBudget

The fleet-shared, Store-backed sibling of tokenBudget. The same stop-at-boundary rule run as an atomic read-modify-write against a shared counter, so one budget L is enforced across every gateway with per-token overshoot of 0 independent of fleet size.

import { distributedTokenBudget } from "throttlekit";

function distributedTokenBudget(options: DistributedTokenBudgetOptions): DistributedTokenBudgetMeter

adaptiveThrottle

Google SRE client-side adaptive throttling. Sheds requests locally before they leave the client when the backend’s accept rate drops.

import { adaptiveThrottle } from "throttlekit";

function adaptiveThrottle(options?: AdaptiveThrottleOptions): AdaptiveThrottle

number

Acceptance multiplier K from the SRE formula. Default 2. Higher values tolerate more backend rejection before shedding locally. Must be ≥ 1.

windowMs

number

Rolling accounting window width in ms. Default 10_000.

clock

Clock

Injected clock. Defaults to the system clock.

random

() => number

Source of randomness for the probabilistic shed. Inject a seeded PRNG for deterministic tests. Default Math.random.

const throttle = adaptiveThrottle({ k: 2 });

async function callBackend() {
  const shouldSend = throttle.request(); // probabilistically shed when backend is overloaded
  if (!shouldSend) throw new Error("shed locally");
  try {
    const result = await backend.call();
    throttle.record(true);
    return result;
  } catch {
    throttle.record(false);
    throw;
  }
}

request

(priority?: number) => boolean

Decide whether to send the next request. Returns true to send, false to shed. priority in [0, 1] scales the shed probability by (1 - priority).

record

(accepted: boolean) => void

Feed back the backend’s outcome. Call only for sent requests (not shed ones).

rejectProbability

() => number

The current local reject probability p in [0, 1]. Read-only.

stats

() => { requests: number; accepts: number; rejectProbability: number }

A point-in-time snapshot for metrics and introspection: rolling request and accept counts plus the current reject probability.

weightedFairShare

Weighted equal-share fairness across tenants. One global budget per epoch-aligned window, split proportionally to per-tenant weights.

import { weightedFairShare } from "throttlekit";

function weightedFairShare(options: WeightedFairShareOptions): WeightedFairShareLimiter

limit

number

required

Global admissions budget shared across all tenants per window.

windowMs

number

required

Window width in ms. Windows are epoch-aligned.

weightOf

(tenant: string) => number

Per-tenant weight. Default () => 1 (equal — equivalent to fairShare).

clock

Clock

Injected clock. Defaults to the system clock.

weightedMaxMin

Weighted max-min fair allocation of an integer limit across tenants with known demands. Work- conserving and weight-honoring. Returns integer credits per tenant.

import { weightedMaxMin } from "throttlekit";

function weightedMaxMin(
  demands: readonly number[],
  weights: readonly number[],
  limit: number
): number[]

Use this for batch allocation (when you have all demands at once) as opposed to the streaming weightedFairShare.

fairShare

Equal-share fairness across tenants — an online approximation of max-min fair allocation. One global budget of limit admissions per epoch-aligned window is split so no single tenant can monopolize it.

import { fairShare } from "throttlekit";

function fairShare(options: FairShareOptions): FairShareLimiter

limit

number

required

Global admissions budget shared across all tenants per window.

windowMs

number

required

Window width in ms. Windows are epoch-aligned: floor(now/windowMs)*windowMs.

clock

Clock

Injected clock. Defaults to the system clock.

checkSync

(tenant: string, cost?: number) => Decision

Synchronous, zero-await check for tenant with the given cost (default 1).

check

(tenant: string, cost?: number) => Promise<Decision>

Promise-returning form of checkSync; resolves synchronously.

reset

(tenant?: string) => void

Reset one tenant’s usage (it leaves the active set), or — with no argument — the whole window.

guaranteedShare

Compute the guaranteed weighted share floor(w_i · limit / W) for each tenant given their weights and a total limit. Returns the static floor a weighted max-min split never drops a backlogged tenant below.

import { guaranteedShare } from "throttlekit";

function guaranteedShare(weights: readonly number[], limit: number): number[]

criticalFractile

The critical-fractile quantile level τ = overrunCost / (holdCost + overrunCost) — the cost quantile that minimises the asymmetric newsvendor / pinball loss, and the target learnedReservation descends onto.

import { criticalFractile } from "throttlekit";

function criticalFractile(holdCost: number, overrunCost: number): number

learnedReservation

Online newsvendor learner for per-request token reservations. Descends onto the cost-optimal criticalFractile quantile with O(√T) regret. @experimental.

import { learnedReservation } from "throttlekit";

function learnedReservation(options: LearnedReservationOptions): LearnedReservation

holdCost

number

required

Penalty per token reserved but unused. Must be > 0.

overrunCost

number

required

Penalty per token of realised cost beyond the reservation. Must be > 0.

maxReservation

number

required

Upper clamp on the reservation (the per-request cap m). Must be > 0.

minReservation

number

Lower clamp on the reservation. Default 0.

const meter = tokenBudget({ budget: 100_000, windowMs: 60_000 });
const policy = learnedReservation({ holdCost: 1, overrunCost: 4, maxReservation: 4096 });

if (policy.reserve() <= meter.remaining()) {
  let produced = 0;
  for await (const tok of completion) {
    if (!meter.debitSync(1).allowed) break;
    emit(tok);
    produced++;
  }
  policy.observe(produced); // learn from realised cost
}

predictiveReservation

Learning-augmented reservation. Blends a per-request output-length prediction with the robust learnedReservation via a Hedge meta-learner: accurate predictions drive cost toward the clairvoyant optimum; adversarial ones fall back to the no-regret quantile. @experimental.

import { predictiveReservation } from "throttlekit";

function predictiveReservation(options: PredictiveReservationOptions): PredictiveReservation

withAnalytics

Wrap a Limiter to track its traffic in-process with Space-Saving top-K heavy hitter detection. Zero config, no OpenTelemetry required. @experimental.

import { withAnalytics } from "throttlekit";

function withAnalytics(limiter: Limiter, options?: AnalyticsOptions): AnalyticsLimiter

topK

number

How many heavy hitters each summary tracks. Default 10.

windowMs

number

Fixed, epoch-aligned window width in ms. Default 60_000.

clock

Clock

Injected clock.

const tracked = withAnalytics(limiter, { topK: 20, windowMs: 60_000 });
const snap = tracked.analytics();
console.log(snap.denyRate, snap.topDenied);

tapDecisions

Wrap a Limiter so a callback fires once per completed check. The tap never breaks the limiter — exceptions inside it are caught and dropped.

import { tapDecisions } from "throttlekit";

function tapDecisions(limiter: Limiter, onDecision: DecisionTap): Limiter

const limiter = tapDecisions(
  rateLimit({ strategy: gcra({ limit: 100, periodMs: 60_000 }) }),
  (event) => {
    metrics.histogram("check_duration_ms", event.durationMs);
    if (!event.decision.allowed) logger.warn({ key: event.key }, "rate limited");
  }
);

sketchRateLimit

Approximate, fixed-memory rate limiter over an unbounded key universe backed by a Count-Min Sketch. Memory is O(1/epsilon · ln(1/delta)) — independent of key count. Never over-admits (hard, non-probabilistic guarantee). @experimental.

import { sketchRateLimit } from "throttlekit";

function sketchRateLimit(options: SketchRateLimitOptions): SketchRateLimiter

limit

number

required

Maximum requests admitted per key within each window.

windowMs

number

required

Window width in ms. Epoch-aligned.

epsilon

number

Additive accuracy: overestimates a key’s count by at most epsilon * N. Default 0.01.

delta

number

Failure probability for the epsilon bound. Default 0.001.

conservative

boolean

Use the Estan–Varghese conservative-update rule for tighter estimates. Default true.

seed

number

32-bit hash seed. Defaults to a per-instance random value (recommended). Pass a fixed value only for reproducible tests.

// Protect against a DDoS flood of millions of unique IPs in fixed memory
const sketch = sketchRateLimit({ limit: 100, windowMs: 60_000, epsilon: 0.01 });
const d = sketch.checkSync(clientIpKey);

multiRateLimit / all / any

Multi-dimensional limiter: evaluate per-IP ∧ per-user ∧ per-route (etc.) atomically. On a synchronous store reads all dimensions, decides, then commits all-or-none. On Redis fuses every dimension into a single Lua round trip.

import { multiRateLimit, all, any } from "throttlekit";

function all<Ctx>(dimensions: Dimensions<Ctx>): MultiStrategy<Ctx>
function any<Ctx>(dimensions: Dimensions<Ctx>): MultiStrategy<Ctx>
function multiRateLimit<Ctx>(options: MultiRateLimitOptions<Ctx>): MultiLimiter<Ctx>

all(dimensions) — allow only if every dimension allows. Consume nothing unless all allow (no partial consume).
any(dimensions) — allow if any dimension allows. Consume only the dimensions that individually allow.

import { multiRateLimit, all, gcra, tokenBucket } from "throttlekit";

type Ctx = { ip: string; userId: string };

const limiter = multiRateLimit({
  strategy: all({
    byIp: {
      key: (ctx) => ctx.ip,
      strategy: gcra({ limit: 100, periodMs: 60_000 }),
    },
    byUser: {
      key: (ctx) => ctx.userId,
      strategy: gcra({ limit: 30, periodMs: 60_000 }),
    },
  }),
  store: redisStore,
});

const d = await limiter.check({ ip: "1.2.3.4", userId: "user:42" });

Core API

Subpath Exports

Admission Control and Weighted Fairness API Reference

unifiedAdmission

UnifiedAdmitter Methods

UnifiedAdmission Fields

tokenBudget

distributedTokenBudget

adaptiveThrottle

weightedFairShare

weightedMaxMin

fairShare

guaranteedShare

criticalFractile

learnedReservation

predictiveReservation

withAnalytics

tapDecisions

sketchRateLimit

multiRateLimit / all / any

Build docs developers (and LLMs) love

Core API

Subpath Exports

Documentation Index

​unifiedAdmission

​UnifiedAdmitter Methods

​UnifiedAdmission Fields

​tokenBudget

​distributedTokenBudget

​adaptiveThrottle

​weightedFairShare

​weightedMaxMin

​fairShare

​guaranteedShare

​criticalFractile

​learnedReservation

​predictiveReservation

​withAnalytics

​tapDecisions

​sketchRateLimit

​multiRateLimit / all / any

Build docs developers (and LLMs) love

unifiedAdmission

UnifiedAdmitter Methods

UnifiedAdmission Fields

tokenBudget

distributedTokenBudget

adaptiveThrottle

weightedFairShare

weightedMaxMin

fairShare

guaranteedShare

criticalFractile

learnedReservation

predictiveReservation

withAnalytics

tapDecisions

sketchRateLimit

multiRateLimit / all / any