Event Sourcing Overview
Event sourcing is a persistence pattern where state changes are stored as a sequence of events rather than by updating records in place. The current state of any entity is reconstructed by replaying its events from the beginning. Benefits for durable workflows:- Complete audit trail: Every state change is recorded with its timestamp and context
- Debugging: Replay the exact sequence of events that led to any state
- Consistency: Events provide a single source of truth for all entity state
- Recoverability: State can be reconstructed from the event log after failures
- Runs: Workflow execution instances (materialized in storage)
- Steps: Individual atomic operations within a workflow (materialized in storage)
- Hooks: Suspension points that can receive external data (materialized in storage)
- Waits: Sleep or delay operations (materialized in storage)
Entity Lifecycles
Each entity type follows a specific lifecycle defined by the events that can affect it. Events transition entities between states, and certain states are terminal—once reached, no further transitions are possible.Run Lifecycle
A run represents a single execution of a workflow function. Runs begin inpending state when created, transition to running when execution starts, and end in one of three terminal states.
Run states:
pending: Created but not yet executingrunning: Actively executing workflow codecompleted: Finished successfully with an output valuefailed: Terminated due to an unrecoverable errorcancelled: Explicitly cancelled by the user or system
Step Lifecycle
A step represents a single invocation of a step function. Steps can retry on failure, either transitioning back topending via step_retrying or being re-executed directly with another step_started event.
Step states:
pending: Created but not yet executing, or waiting to retryrunning: Actively executing step codecompleted: Finished successfully with a result valuefailed: Terminated after exhausting all retry attempts
Hook Lifecycle
A hook represents a suspension point that can receive external data. Hooks can receive multiple payloads while active and are disposed when no longer needed. Hook states:active: Ready to receive payloads (hook exists in storage)disposed: No longer accepting payloads (hook is deleted from storage)conflicted: Hook creation failed because the token is already in use by another workflow
Wait Lifecycle
A wait represents a sleep operation. Waits track when a delay period has elapsed. Wait states:waiting: Delay period has not yet elapsedcompleted: Delay period has elapsed, workflow can resume
Event Types Reference
Events are categorized by the entity type they affect. Each event contains metadata including a timestamp and acorrelationId that links the event to a specific entity.
Run Events
Step Events
Hook Events
Wait Events
Event Consumption and Replay
TheEventsConsumer class manages the sequential processing of events during workflow replay. It implements a subscription-based model where different parts of the workflow runtime can subscribe to specific event types.
How it works:
- Sequential Processing: Events are consumed in order, one at a time
- Callback Subscription: Components subscribe with callbacks that inspect each event
- Result Handling: Callbacks return:
Consumed: Event processed, continue to next eventNotConsumed: Event doesn’t match, try next callbackFinished: Event processed, remove this callback
Entity IDs
All entities in the Workflow DevKit use a consistent ID format: a 4-character prefix followed by an underscore and a ULID (Universally Unique Lexicographically Sortable Identifier).
Why this format?
- Prefixes enable introspection: Given any ID, you can immediately identify what type of entity it refers to
- ULIDs enable chronological ordering: ULIDs encode a timestamp in their first 48 bits, making them lexicographically sortable by creation time
Terminal States and Consistency
Terminal states represent the end of an entity’s lifecycle. Once an entity reaches a terminal state, no further events can transition it to another state. Run terminal states:completed: Workflow finished successfullyfailed: Workflow encountered an unrecoverable errorcancelled: Workflow was explicitly cancelled
completed: Step finished successfullyfailed: Step failed after all retry attempts
disposed: Hook has been deleted from storageconflicted: Hook creation failed due to token conflict
completed: Delay period has elapsed