Long-Horizon Agents | Library

Short definition: A long-horizon agent is an agentic process that continues autonomously across many steps, context changes, and possibly restarts until it reaches a verifiable goal or a stop condition. “Long” does not merely mean a generous timeout. It means the run needs durable state, budgets, checkpoints, and a safe way to resume.

What it is, and what it is not

A chat agent usually reacts to one instruction, makes a few tool calls, and returns an answer. A long-horizon agent receives a goal and works through tens or hundreds of steps over hours or days. It may inspect a repository, edit files, run tests, compare outputs, revise its plan, and wait for an external system or a person.

The boundary is not whether execution happens in the background. An asynchronous agent merely separates submission from delivery: a task enters a queue and the answer arrives later. It might still perform one inference step. A workflow follows a path mostly fixed in code: fetch data, classify it, produce a draft, request approval. A long-horizon agent selects its next action from the current state and can change course when reality differs from the plan. The strongest production design is often a hybrid: deterministic workflow code owns phases and permissions, while an agent makes bounded decisions inside selected phases.

It is not a model left in an endless loop. The model is a replaceable decision component. The task survives because a harness around it owns state, tools, the event queue, budgets, logs, and continuation rules.

Architecture: goal, state, checkpoint

A reliable run separates three layers:

Goal and completion contract. The assignment specifies what to produce, what must not change, and how success will be demonstrated. For a code change, that may include allowed files, passing tests, clean lint, no secret leakage, and a diff confined to scope.
Durable task state. The current plan, completed steps, artifacts, verification results, open questions, consumed budget, and tool audit log live outside the model context. A context window is a workbench, not a database.
Checkpoint. A consistent snapshot from which execution can continue without guesswork. It records the assignment version, plan state, artifact links or hashes, last verified results, pending approvals, and environment identity. “The model remembers where it stopped” is not a checkpoint.

A useful state machine has explicit modes such as planned, executing, verifying, awaiting_approval, blocked, completed, and failed. The harness performs transitions, not a free-form model claim. Every external action gets an idempotency key or equivalent duplicate protection. A recovery can then repeat a safe query rather than issue a second payment or send a second email.

Agentic drift without mythology

Agentic drift is a useful operational label for gradual deviation from the original goal, constraints, or success measure. It is not one precisely standardized diagnosis, and it does not require a story about a willful machine. Most cases are a mundane combination of context compression, local optimization, and badly designed feedback.

An agent encounters a broken dependency. Fixing it becomes the immediate problem, so the agent starts rewriting the library even though the actual assignment was a three-line application change. In another run, a metric becomes a proxy for the goal: the agent maximizes passing tests by weakening a test because that route is easier than fixing the product.

The answer is not a longer prompt. Before each phase, the harness should reload the immutable contract, compare the plan with the allowed scope, constrain actions through permissions, and ask a machine-checkable question: did the latest action move the task toward the declared done state? Checkpoints should also preserve rejected approaches so a restart does not pay to rediscover them.

Budgets and stop conditions

A long run needs several budgets at once:

maximum cost or token usage,
wall-clock duration and active compute time,
number of steps, tool calls, and repeats of the same error,
write, network request, and parallel worker limits,
maximum diff size or number of affected objects,
a human attention budget, including how often the agent may escalate.

Limits should be hard and visible in task state. “Continue until done” is not a stop condition. A run succeeds only when the verification contract passes. It pauses safely for an approval. It ends as blocked or failed after exhausting a budget, repeating the same failure, losing access to a required system, or discovering conflicting requirements. Silently extending a budget turns a design fault into an inference bill.

Resumability is a product capability

Resumability does not mean resending the whole transcript. After a restart, the orchestrator locates the latest confirmed checkpoint, verifies whether the environment has changed, and schedules only missing work. Each step should have inputs, outputs, and a status such as not_started, in_progress, verified, or invalidated.

Artifacts belong in versioned storage. Events belong in an append-only log. Secrets belong in a secret manager, not a checkpoint. The next prompt is assembled from the contract, current state, and relevant evidence. If a person changed the branch, database schema, or goal priority in the meantime, the old checkpoint is invalidated or deliberately migrated. Resuming without environment validation produces confident work against yesterday’s reality.

Verification and human approval

For long tasks, verification is not a final gate. It is navigation. After a small change, the agent runs a cheap local check. After a phase, it runs an integration check. Before completion, it executes full acceptance. For code, this can include tests, lint, type checking, security scanning, diff inspection, and a screenshot or live application exercise. For data work, it can include schema checks, row counts, invariants, samples, and a reproducible report.

The verifier should be as independent from the author as practical. If the same model writes a change and merely states that it is correct, the claim is weak evidence. Deterministic tests, a separate review step, comparison with a reference result, and retained evidence are stronger.

People should not approve every click. Approval gates belong before irreversible or socially consequential actions: a production deploy, message delivery, payment, data deletion, permission expansion, or publication. The request should show intent, the exact diff or payload, check results, risks, and a rollback plan. An “Approve” button without that context transfers liability rather than providing control.

Designing evals for long tasks

A one-shot pass rate is insufficient. The eval suite should contain realistic tasks in clean environments, hidden acceptance tests, and failures that occur in production: a tool timeout, worker restart, dependency change, ambiguous requirement, or pending approval.

Measure at least:

the fraction of tasks that reached the real target state,
time and cost to a verified result, not to the first draft,
steps, repeated failures, and human interventions,
scope violations and unsafe attempts, including blocked ones,
successful recovery from a checkpoint,
evidence quality and the correctness of stop decisions.

Segment results by task length and task type. METR’s “task-completion time horizon” asks how long a task, measured by the time a qualified human needs, can be while an agent still reaches a chosen reliability level. That is more informative than saying an agent “works all day.” A long runtime can indicate persistence, but it can also indicate a slow loop. Evals must measure completion, not activity.

Incident recovery

When a run goes wrong, the first move is not another prompt. The orchestrator stops new work, revokes risky credentials, preserves logs, and identifies the last known good checkpoint. It then separates internal artifacts from external effects: a commit can be reverted; a delivered email cannot. Every tool therefore needs a reversibility class and, where possible, a compensating action.

A practical recovery sequence is: freeze the run, collect a timeline, compare actual changes with allowed scope, restore or compensate external state, fix the root cause in the harness or evals, and only then continue from a new checkpoint. An incident that does not produce a regression test is likely to return.

A practical implementation blueprint

Write a task contract: goal, non-goals, allowed systems, verification commands, and stop conditions.
Divide execution into deterministic phases. Permit agentic choice only where the route cannot be specified in advance.
Store state in a database and artifacts in versioned storage. Record every tool call as an event.
Use short, idempotent steps and checkpoint after each verified milestone.
Enforce budgets and loop detection. Three copies of the same error are not three new attempts.
Add verifiers from cheapest to most expensive. An output without evidence is not completed.
Separate approval gates from ordinary control flow. Waiting is a durable state, not a process blocked in memory.
Test restart, duplicate event delivery, environment change, and rollback before the first production-length run.
Begin in a sandbox with read-only tools. Expand permissions based on eval data and incidents, not the best demo.

Common failure modes

A background job marketed as a long-horizon agent: asynchrony alone adds neither adaptive planning nor resumability.
Transcript as the only state: compression or restart erases decisions, budgets, and artifact identity.
Checkpointing every token: expensive noise. Checkpoint consistent, verified milestones.
One global definition of done: local faults surface too late. Verify by phase.
A dollar budget only: the agent can still exhaust API quotas, human attention, or allowed change scope.
Approval fatigue: a person confirms mechanically without context. Ask less often and provide evidence.
Resume without idempotency: restart duplicates an external effect.
Evaluation by activity: a long log is not a completed task.
Treating drift as a prompt-only problem: tools, permissions, and verifiers must enforce scope.

What to remember

A long-horizon agent is not a smarter chatbot with a larger timeout. It is a durable stateful system in which a model proposes the next action, while the harness owns the goal, history, budget, and right to stop. Start with a workflow and add agentic choice only where it pays. Keep state outside the model, checkpoint verified milestones, design external actions for safe repetition, and measure time to proven results. Autonomy can grow only as fast as verification and recovery capability.

Sources

Building effective agents (Anthropic) - a practical distinction between workflows and agents, common orchestration patterns, and the case for the simplest design that works.
Measuring AI Ability to Complete Long Tasks (METR) - the task-completion time-horizon methodology, including important caveats about what the metric does and does not establish.
12-Factor Agents (HumanLayer) - production-oriented principles for owning context, keeping state outside the model, using small steps, and controlling execution flow.
Codex cloud (OpenAI documentation) - primary documentation for isolated cloud tasks, environments, and background coding-agent work.
Workflow execution (Temporal documentation) - a useful reference model for durable execution, event history, retries, and process recovery.