Aether architecture: hexagonal design and internals

Aether is built around one organizing principle: the engine is a pure scheduler, not an executor. It reads workflow documents, maintains the phase state machine, determines which tasks are ready to run, and dispatches them through a broker — but it never contains a single line of business logic. Every infrastructure concern is expressed as a Go interface, injected at construction time, and enforced at the compiler level by the internal/ package boundary. Understanding this separation is the foundation for understanding every other design decision in Aether.

The engine as pure scheduler

The engine’s job is narrow and precisely bounded:

Determine readiness — for a dag template, a task is ready when all its declared dependencies have reached a terminal phase. For a loop template, the next iteration is ready when the previous one completes and the loop condition still holds.
Dispatch — call broker.TaskBroker.Dispatch with a TaskAssignment that carries all resolved inputs, secrets, and metadata the worker needs.
React — when OnTaskStarted or OnTaskCompleted fires, update phase state, fire lifecycle hooks, and call advanceScope to re-evaluate the graph.

The engine never reads files, never makes network calls, never evaluates expressions directly (that is delegated to expr.Evaluator), and never writes to a log. All of its observable effects pass through the injected interfaces.

PhaseSkipped and PhaseCancelled are set exclusively by the engine. Executors return an integer ExecCode; the engine maps it to a Phase. Users can influence the mapping via phaseConditions expressions, but they cannot set engine-reserved phases directly.

Hexagonal architecture

Aether’s architecture is hexagonal (also called ports-and-adapters). The core engine lives at the center; all infrastructure lives at the edges behind interface ports. Dependencies flow inward only — implementations depend on the engine’s interface definitions, never the reverse.

                        ┌─────────────────────┐
         store.Store ───►                     ◄─── idgen.Generator
    broker.TaskBroker ──►                     ◄─── hook.Notifier
    executor.Plugin ────►    Engine (core)    ◄─── expr.Evaluator
     secret.Provider ───►                     ◄─── artifact.Repository
    timeout.Watcher ────►                     ◄─── vars.Source
                        └─────────────────────┘

Each port is a Go interface in its own package. The package contains only the interface definition and any supporting types — no implementation. Implementations are provided by the caller through functional options (option.go) and are never imported by the core engine package.

The ten extension interfaces

Required interfaces

These four must be provided at construction time. aether.New returns an error if any is absent.

Interface	Package	Responsibility
`store.Store`	`github.com/BabySid/aether/store`	State persistence — workflow runs, task runs, cron records, schemas
`broker.TaskBroker`	`github.com/BabySid/aether/broker`	Task dispatch, cancellation, fetch, start, and completion
`executor.Plugin`	`github.com/BabySid/aether/executor`	Task execution logic — at least one plugin must be registered
`idgen.Generator`	`github.com/BabySid/aether/idgen`	Unique ID generation for workflow runs and task runs

Optional interfaces

These are provided through functional options. The engine behaves identically without them, with features simply unavailable.

Interface	Option	What it enables
`expr.Evaluator`	`WithExprEvaluator`	`when` conditions, `repeatCondition`, `phaseConditions` expressions
`hook.Notifier`	`WithHookNotifier`	Workflow and task lifecycle event notifications
`artifact.Repository`	`WithArtifactStore`	Artifact upload/download (interface wired, execution integration pending)
`secret.Provider`	`WithSecretStore`	Secret injection into task inputs at dispatch time
`timeout.Watcher`	`WithTimeoutWatcher`	Deadline expiry detection — calls `OnTaskTimeout`/`OnWorkflowTimeout`
`vars.Source`	`WithVarsSource`	Variable injection (e.g., `system.os`, `system.arch`, custom namespaces)

Two additional optional interfaces extend engine capabilities further:

Interface	Option	What it enables
`cron.Scheduler`	`WithCronScheduler`	`CronWorkflow` submission, triggering, and management
`worker.Registry`	`WithWorkerRegistry`	Worker registration and capability discovery for distributed brokers

Wiring example

The minimal required wiring, taken directly from the playground’s buildEngine function:

eng, err := aether.New(
    aether.WithStore(memStore),
    aether.WithExecutorRegistry(reg),   // pre-built registry with plugins
    aether.WithIDGenerator(NewAtomicIDGen()),
    aether.WithTaskBroker(brok),
)

Adding optional interfaces:

eng, err := aether.New(
    aether.WithStore(memStore),
    aether.WithExecutorRegistry(reg),
    aether.WithIDGenerator(NewAtomicIDGen()),
    aether.WithTaskBroker(brok),
    aether.WithExprEvaluator(NewSimpleEvaluator()),
    aether.WithTimeoutWatcher(newPollingWatcher(memStore, 500*time.Millisecond)),
    aether.WithVarsSource(&vars.SystemSource{}),
    aether.WithHookNotifier(myNotifier),
    aether.WithSecretStore(mySecretStore),
)

WithExecutor and WithExecutorRegistry are two ways to register the same thing. Use WithExecutor to register plugins one at a time; use WithExecutorRegistry when you already have a pre-built *executor.Registry that you share with the broker or worker layer.

The scope-tree scheduling model

Every workflow execution forms a tree of TaskRun records connected by ParentRunID. This tree is the “scope tree.” Understanding it explains how any level of nesting schedules correctly with the same algorithm. Root scope. When a workflow is submitted, the engine creates a single root TaskRun for the entrypoint template. If the entrypoint is a dag, that TaskRun becomes the parent for all tasks in the DAG. Child scopes. When a dag task references another template (which may itself be a dag, task, or loop), a new child TaskRun is created with ParentRunID set to the parent DAG’s TaskRun. This nesting can be arbitrarily deep (up to maxNestedDepth, default 3, max 10). advanceScope. After any task completes, the engine calls advanceScope(parentRunID). This function:

Loads all TaskRun records sharing the given parentRunID
Finds tasks whose dependencies are all terminal — these are newly ready
Dispatches each ready task to the broker
If all tasks in the scope are terminal, the scope itself is finalized
Walks upward to the parent’s parentRunID and repeats

This recursive walk means a deeply nested DAG inside a loop inside another DAG all converges through the same advanceScope function. There are no special cases for nesting depth.

WorkflowRun (root)
└── TaskRun: entrypoint-dag (parentRunID="")
    ├── TaskRun: fetch-data (parentRunID=entrypoint-dag)
    ├── TaskRun: transform-data (parentRunID=entrypoint-dag, deps=[fetch-data])
    └── TaskRun: notify (parentRunID=entrypoint-dag, deps=[transform-data])

When fetch-data completes, advanceScope(entrypoint-dag) finds that transform-data’s only dependency is now Succeeded, dispatches it, then returns. When transform-data completes, the same function dispatches notify. When notify completes, all tasks in entrypoint-dag’s scope are terminal, so entrypoint-dag itself is finalized and the workflow completes.

Fat task assignments

Workers never query the store.Store. This is a deliberate constraint, not an oversight. When the engine dispatches a task, it builds a broker.TaskAssignment that carries everything the worker needs: resolved input parameters, executor type and schema, task and template names, retry count, timeout, and secret references. The worker receives this struct from the broker, calls executor.Plugin.Execute, and reports the result back through broker.CompleteTask. It never needs to look up the workflow or any sibling task’s state. This design eliminates a class of distributed coupling problems. Workers can run in isolated processes, in separate data centers, or without any network access to the engine’s state store. They are simple, stateless, and independently testable.

Phase ownership

The engine is the sole writer of Phase values on WorkflowRun and TaskRun records. The phase state machine is:

Created → Ready → Running → Succeeded
                          → Failed
                          → Error
                          → Timeout
                          → Skipped
                          → Cancelled
                          → Suspended  (task only; resumes to Running)

Executors return an ExecCode integer. The engine maps ExecCode to Phase:

`ExecCode`	Default phase
`ExecCodeSucceeded` (0)	`Succeeded`
`ExecCodeFailed`	`Failed`
`ExecCodeError`	`Error`
`ExecCodeTimeout`	`Timeout`
`ExecCodeSuspended`	`Suspended`

Users can influence the mapping with phaseConditions expressions on a task declaration. For example, a task can treat ExecCodeFailed as Succeeded when a specific output condition holds. However, PhaseSkipped (set when a when condition evaluates to false) and PhaseCancelled (set by engine.Cancel) are exclusively engine semantics. Executors cannot return these phases.

The `internal/` boundary

All scheduling algorithms, binding logic, validation, retry/timeout mechanics, and DAG traversal live under internal/. The Go compiler enforces this boundary: no code outside the github.com/BabySid/aether module can import internal/ packages. This means implementation details can never leak into the public API, and callers can never depend on internal behavior. Outside of internal/, every package (except the top-level aether package itself) is a pure interface layer — it defines a contract and supporting types, nothing else. This makes each interface package a stable, independently versioned contract. Key internal/ subsystems:

Subsystem	File(s)	Responsibility
DAG traversal	`dag.go`	`FindReadyTasks`, `FindTemplate`, `FindTask`
Parameter binding	`binding/`	`{{inputs.parameters.name}}` interpolation, expression environment building
Validation	`validate.go`	Structural and semantic workflow validation before any state is persisted
Retry	`retry.go`	`ShouldRetry` — evaluates retry policy against current phase and retry count
Timeout	`timeout.go`	Deadline arithmetic and watchdog integration
Merge	`merge.go`	Output parameter merging (executor result wins over template-declared values)
Phase	`phase.go`	`EvalPhaseConditions` — maps `ExecCode` to `Phase` with optional expression override
Hooks	`hooks.go`	`FireWorkflowHooks`, `FireResumeHook` — dispatches to `hook.Notifier`
Defaults	`defaults.go`	`FillDefaults`, `FillCronDefaults` — applies missing field defaults before validation

Engine public API surface

Engine is the only struct exposed to callers. Its API is intentionally small:

Method	Description
`New(...Option) (*Engine, error)`	Construct and validate the engine
`Start(ctx) error`	Launch background services (timeout watchdog, cron scheduler)
`Stop()`	Shut down all background services; safe to call multiple times
`Submit(ctx, *model.Workflow) (string, error)`	Parse, validate, and dispatch a workflow; returns run ID
`SubmitCronWorkflow(ctx, *model.CronWorkflow) (string, error)`	Register a cron-triggered workflow
`Get(ctx, workflowID) (*WorkflowExecution, error)`	Read current execution state (non-blocking)
`Resume(ctx, workflowID, taskID, payload) error`	Re-dispatch a suspended task with merged payload
`Cancel(ctx, workflowID) error`	Cancel all non-terminal tasks and mark the workflow cancelled
`OnTaskStarted(ctx, taskRunID)`	Lifecycle callback: broker calls this when a worker begins a task
`OnTaskCompleted(ctx, *broker.TaskResult)`	Lifecycle callback: broker calls this when a worker finishes a task

Everything else is opt-in through functional options. There are no global singletons, no init() side effects, and no hidden configuration files.

Executor plugin

Implement the executor.Plugin interface for your own task execution logic

Store interface

Build a persistent store.Store backed by your database of choice

Broker interface

Implement broker.TaskBroker for distributed or queue-based task dispatch

Other interfaces

expr.Evaluator, hook.Notifier, secret.Provider, timeout.Watcher, and more

Get Started

Core Concepts

Guides

Extension Points

Aether architecture: hexagonal design and internals

The engine as pure scheduler

Hexagonal architecture

The ten extension interfaces

Required interfaces

Optional interfaces

Wiring example

The scope-tree scheduling model

Fat task assignments

Phase ownership

The `internal/` boundary

Engine public API surface

Executor plugin

Store interface

Broker interface

Other interfaces

​The engine as pure scheduler

​Hexagonal architecture

​The ten extension interfaces

​Required interfaces

​Optional interfaces

​Wiring example

​The scope-tree scheduling model

​Fat task assignments

​Phase ownership

​The internal/ boundary

​Engine public API surface

Executor plugin

Store interface

Broker interface

Other interfaces

The engine as pure scheduler

Hexagonal architecture

The ten extension interfaces

Required interfaces

Optional interfaces

Wiring example

The scope-tree scheduling model

Fat task assignments

Phase ownership

The `internal/` boundary

Engine public API surface