Runs

A run (workflow instance) is one execution of a workflow by an agent. Queueing a run returns immediately; execution proceeds in the background on a durable queue, and everything about it is observable.

Launching a run

PUT /api/workflowinstances/run (permission run-workflow-instance) answers 202 Accepted with { workflowInstanceId, alreadyQueued }.

Field	Purpose
workflowId	The workflow to run.
agentId	The agent that runs it. Omit it to use the workflow's default agent.
initialContext	Key/value run inputs, including the reserved task_id and purpose.
workflowVersionNumber / useDraft	Pin a published version, or test the draft. Omit both for the latest published version. Not both at once.
clientRequestId	Up to 128 characters, unique per tenant. Re-sending a launch with the same id returns the run it already queued (alreadyQueued: true) instead of starting a second one, so a double click or a client retry is safe.
dryRun	Check that the launch would be accepted (request, workflow, version and agent) without queueing anything. The response has no run id.
callback	{ url } to receive the run's status by HTTP. See Following a run from outside.

Runs also start from the console's Run dialog, from schedules and triggers, as child runs of sub-workflows, and from a conversation thread whose Orchestrator agent calls a workflow tool. A run launched from a thread reports the thread as threadId on its detail (GET /api/workflowinstances/{id}/detail). The run belongs to the person who started it, and its cost stays on the run, not the thread.

Lifecycle

Idle ──▶ Running ──▶ Completed
  │        │  ▲
  │        │  └──── Waiting        (paused on a wait, a confirmation or a question)
  │        │
  │        └──▶ Failed ──▶ AwaitingRetry ──▶ Running …
  │                 └────▶ DeadLettered     (no attempts left)
  └──▶ Cancelled   (from Idle, Running, Waiting or AwaitingRetry)

State	Meaning
Idle	Queued, not started yet.
Running	An attempt is executing.
Waiting	Paused until a signal arrives. waitKind says what for: ReviewOutcome, Clarification or ToolConfirmation.
Completed	Every step completed or was skipped.
Failed	An attempt failed, with the reason recorded. A retry may follow.
AwaitingRetry	Waiting for its backoff before the next attempt.
DeadLettered	Out of attempts. The run is parked for an operator, and a fallback workflow can start.
Cancelled	Cancelled before it ended.

Retries, attempts and dead letters

A failed attempt is retried automatically with backoff until the run runs out of attempts, then dead-lettered.

Setting (WorkflowExecution:Retry:)	Default
MaxAttempts	3	Attempts per run, including the first. A workflow can override it (1–10) with PUT /api/workflows/retry-policy. 1 means no automatic retry.
BackoffStrategy	Exponential	Fixed, Linear or Exponential.
BaseDelay / MaxDelay	30 s / 15 min	The backoff unit and its ceiling.

A retry does not start over. The run keeps a durable cursor of completed steps and continues from the first step that had not finished, and a for-each step reuses the items that had already succeeded. The run's attemptCount records how many attempts it used.

Leases and crash recovery. A host that picks up a run holds a lease on it (RunQueue:ClaimVisibilityTimeoutSeconds, default 300) and renews it while the run is live. If the host crashes, the lease expires and another host takes the run over. Step instances left running by the crashed attempt are marked failed with a reason, so the history stays truthful. A host that loses its lease stops its in-flight work, so two attempts never write at once. Each host runs at most WorkflowExecution:MaxConcurrentExecutions runs at once (default 4), and the rest wait in the queue.

Cancelling a run

PUT /api/workflowinstances/cancel with { workflowInstanceId } (permission cancel-workflow-instance) accepts any run that has not ended: Idle, Running, Waiting or AwaitingRetry. A run that has already ended answers 409.

• A queued or awaiting-retry run is dropped and never starts.
• A running run is signalled to stop at its next checkpoint.
• A waiting run's pending confirmations and questions are cancelled.
• Cancelling a run cancels all its running child runs. Cancelling only a child fails the parent step that started it.

What a step execution involves

For each step, in order:

1. The step's condition, if any, is evaluated. A false condition skips the step (see Step conditions).
2. An agent instance is created: the identity-bearing execution shell for this step.
3. A token exchange delegates the run's principal to the agent, restricted to each tool's resource (audience) and scope. See Identity & Security.
4. A step instance runs the model against the step instruction plus the current visible context, after any retrieval calls.
5. The model may call the tools on the step's allowlist, within the step's round and token limits. Tool calls carry the delegated token.
6. Outputs are promoted into the context channel, the instances complete, and the next step begins.

A step failure fails the attempt, and later steps do not execute. The run is then retried or dead-lettered as described above.

Tool calls that need confirmation

Each tool on a step's allowlist runs either automatically (the default) or only after a person confirms it. Use confirmation for tools that change things (filing, merging, sending), and leave read-only tools automatic.

When the model calls a tool that needs confirmation:

1. The run pauses (state Waiting) with the proposed call: the tool and its arguments.
2. A reviewer opens it on the Confirmations page, or on the run's page, and either approves it, optionally editing the arguments first, or rejects it with a reason.
3. The run continues the same step from where it stopped:
   • an approved call runs (with the edited arguments, when changed);
   • a rejected call never runs, and the model is told why and carries on without it.

A call no one decides in time is declined by default (ToolConfirmation:TimeoutHours, 72). Deciding needs confirm-tool-calls, and the rest of the run continues as the reviewer. Every request and decision is recorded in the run's activity feed. Calls in for-each items, parallel branches and child runs cannot pause the run, so they are declined there. See the tool confirmations runbook.

Asking a person for clarification

A model step can allow clarification. Its model is then offered a built-in tool, request_clarification, for the moments it cannot go on responsibly without a person: an ambiguous instruction, a missing fact, or a choice that is theirs to make. The model asks one question and may suggest likely answers.

When it asks:

1. The run pauses (state Waiting, waiting on a clarification) with the question.
2. Someone answers on the Clarifications page, or on the run's page. They pick a suggested answer or write their own.
3. The run continues the same step: the answer is the tool's result, and the model carries on with it.

A question no one answers in time is, by default, handed back to the model as unanswered, and the model continues on its best judgement. An administrator can instead make an unanswered question fail the run, remind people, or escalate. Answering needs answer-clarifications, and the rest of the run continues as the person who answered. Questions and answers are recorded in the run's activity feed and announced to the portal. Steps in for-each items, parallel branches and child runs cannot pause the run, so there the model is told no one can be asked. See the clarifications runbook.

The context channel

The run's context is its only cross-step data channel. Every entry has a key, a value, and a provenance kind:

Kind	Meaning
Input	Supplied by the caller when the run was queued, or a wait's outcome.
ToolOutput	A tool's deterministic result: the authoritative data channel.
StructuredOutput	A step's schema-validated output, or a for-each step's aggregate.
ModelText	The model's own text: a summary or narrative, not authoritative data.

Step outputs are promoted under predictable keys, numbered by the step's 1-based execution order:

• toolResult_<n>: the step's last tool result (only when the step made a tool call and the workflow step promotes tool results, which is the default).
• structuredOutput_<n>: the step's validated output, when it has an output schema; for a for-each step, the per-item aggregate; for a sub-workflow step, the mapped child outputs.
• iterationSummary_<n>: a for-each step's item counts and failed items.
• modelResponse_<n>: the step's final model text.

Later steps see the most recent value per key, so instructions can reference earlier results by these keys (e.g. "using the match verdict in toolResult_1…"). Large values appear in the prompt as a short preview, which the model reads in full with read_context.

Following a run from outside

• Callbacks. A launch with callback.url gets signed HTTP POSTs for started, completed, failed and cancelled, each at most once per run and retried until delivered. The URL must belong to a callback target configured under RunCallbacks:Targets. See the run callbacks contract.
• Run events. With RunEvents:Enabled, every run's lifecycle is published to a Service Bus topic (default agentfleet-run-events): RunQueued, RunStarted, RunWaiting, RunCompleted, RunFailed, RunDeadLettered and RunCancelled, with step progress, tokens, cost, trigger and version. Delivery is at least once, so dedupe on eventId. See the run events contract.

Observability

• Run page. Summary (state, step progress, tokens, cost, timing, version, trigger, purpose), the Path over the workflow graph, a Timeline of steps with their model rounds and tool calls on one time axis, and the run's confirmations and questions.
• Model calls. Every LLM invocation with input, output, token counts, latency, finish reason, and round.
• Tool calls. Every tool invocation per step, including retrieval calls and built-ins.
• Activities. The audit feed of lifecycle events across the platform.
• Tracing and metrics. With Telemetry:Enabled, Agentfleet exports OpenTelemetry traces and metrics over OTLP. One run is one trace, and its trace id is the run id without dashes. Set Telemetry:TraceUrlTemplate to link the Timeline to your trace backend. See the observability runbook.

Run operations

The Run operations page is where operators look after the queue:

Section	API	Permission
Queue health: ready, delayed, claimed and dead-lettered counts	GET /api/runoperations/queue	view-run-queue
Dead-lettered runs, with Requeue (a fresh set of attempts)	GET /api/runoperations/dead-letters, POST /api/runoperations/dead-letters/requeue	view-dead-lettered-runs, requeue-dead-lettered-run
Paused runs and what each waits for, with Resume	GET /api/runoperations/paused, POST /api/runoperations/paused/resume	view-waiting-instances, resume-workflow-instance

Resuming takes { workflowInstanceId, correlationKey, outcome }. The outcome is written to the wait's outcome key. Answer questions and decide confirmations on their own pages instead, so the model gets the answer.

Related

• Run Inputs: including the reserved task_id and purpose keys.
• Workflow Design: conditions, loops, parallel branches, waits and sub-workflows.
• Automation: starting runs on a schedule, from an event or from a webhook instead of by hand.
• Evaluations: scoring a workflow's runs against expected cases.