Agents — when an LLM gets hands and memory

Golden rule: Workflow first. An agent only when nothing simpler works. Autonomy is not a feature; it is a cost in tokens, latency and unpredictability. Buy it only for tasks that need exploration: unknown data, unknown environment, unknown number of steps.

What it is

An agent = an LLM in a loop with tools (search, shell, APIs) and some form of memory. The cycle is dumb:

while not done:
    decide(next action) → call tool → read result → update plan

That is it. Everything else is a variation. And what it is not: not consciousness, not AGI, not one giant prompt. State, plan and memory live outside the LLM, in the application. The model only picks the next action.

Agent vs workflow — the decision that matters

Workflow has predefined steps; the LLM fills holes (classic “summarize → translate → send”). Agent picks the next step from state. Workflow is cheaper, safer and easier to debug. Agents handle tasks whose path you cannot fully specify. Most production “agents” are workflows with one agentic step — and that is correct.

Closed-loop: without verification, autonomy is expensive random walk

One-shot generation (model writes, human finishes) feels fast and scales poorly. Closed-loop means after an action the agent runs a check — test, lint, SQL assert, screenshot, CI — and continues or repairs, ideally without a human in every step.

Verification is not a gate bolted on at the end. It is the loop’s sense of “done”. Without a machine-checkable finish condition the agent does not know when to stop and you do not know when you may grant more autonomy. For coding agents that means tests and CI before fleets of background jobs. For business agents it means assertable outcomes: record exists, email is a draft awaiting approval, amount matches.

Agent speed is not quality’s enemy. Speed is encoded quality: evals, skills, instruction files, narrow tools, logs. What you cannot run automatically, you cannot scale.

Build an agent step by step

1. Try without an agent. One solid prompt. Then a fixed workflow. Only then a loop. Each step down is an order of magnitude cheaper to run and debug.

2. Define “done” mechanically. Tests passed, record found, file exists. A loop without a stop condition is a cost generator. This is the core of closed-loop.

3. Start with 3–5 tools. Fewer tools, better choices. Write tool descriptions like docs for a new teammate: what it does, when to use it, what it returns. Bad tool docs are the silent cause of dumb agents.

4. Set limits. Max steps, token budget, timeout. A looping agent should fail loudly, not burn the budget. Log every action or you will never know why it did what it did.

5. Human-in-the-loop for irreversible actions. Deletes, payments, deploys, outbound data need a human. Expand autonomy from logs and success rate, not from demo hype.

6. Layer memory deliberately. Working memory (current window), episodic (past runs via log + summary), semantic (domain facts via structured store + lookup). There is no single magical memory layer. That is marketing.

7. Measure success rate and actions per task. If you do not know average steps and success rate, you are hoping, not operating. An eval set with known outcomes matters as much as in RAG. Track cost and human-intervention rate too.

Glossary

• Tool use — the format for “call X with args Y”. Solved at major providers.
• MCP (Model Context Protocol) — a standard way to expose tools and resources as plugins.
• ReAct — original “reason then act” paper; now a historical name for the agent loop.
• Closed-loop — agent + automatic verification in the loop, not one-shot output plus human last-mile on every step.

Common mistakes

• Agent for a workflow task → step down a level.
• No loop limits → max steps + budget + timeout, fail loud.
• Too many tools → under 10, ideally 3–5; route or split into specialists.
• No human gate on irreversible actions → approvals; autonomy is earned.
• State stuffed into the prompt → keep plan and intermediates in the app; put only the next-step needs in context.
• Judging by demos → demos are selected wins; decide on eval success rate.
• Autonomy without verification → pass/fail signal first, then background jobs and automerge.

When not to use it

When you can write fixed steps — workflow wins. When the task is one-off and cheaper by hand. When you cannot verify results: an agent without verification is a costly random walk through your system.

Sources

• Building effective agents (Anthropic)
• 12-Factor Agents
• Claude Code: Best practices for agentic coding (Anthropic)
• AI Engineering (Chip Huyen)
• METR

What to remember

An agent is a dumb loop around a smart model. Most problems people blame on the model (loops, lost context, bad choices) are loop bugs. Build from simplest: prompt → workflow → agent. Define done mechanically, limit tools and steps, log everything, dose autonomy from data. Without verification you are not buying an agent. You are buying an action generator.