What an agent is

The loop is the definition

An agent is a program that repeats three steps: look at the current state, choose an action, observe the result. The action can be a tool call, a search, a message to another agent or a final answer. The loop ends when the model says it is done or the budget runs out.

Nothing about that requires a particular framework, and nothing about it requires a language model. A thermostat is an agent in this sense. What the model adds is the ability to choose among open-ended actions described in words.

The state is the part people underestimate. It holds the question, the observations so far, and whatever the plan or budget needs. The model does not remember the previous iteration; the loop sends the state again. That is why the quality of the state — what is kept, what is summarised, what is dropped — decides more about the agent than the wording of its prompt.

Why a chain is not enough

A chain knows every step before it runs. That is fine when the steps are fixed and wrong when they depend on what was found: you cannot decide to look up a second document until you have read the first.

The moment a workflow needs “if the search returns nothing, try a different query”, it needs a loop. That is the honest threshold for calling something an agent.

The smallest agent

state = {"question": question, "steps": []}

for step in range(max_steps):
    action = model.decide(state)          # "search", "read", "answer"
    if action.kind == "answer":
        return action.text
    result = run_tool(action)             # untrusted input, validated first
    state["steps"].append({"action": action, "result": result})

return "stopped: step limit reached"

The trace is the state

Each iteration should append one record: the action the model chose, the arguments it proposed, the decision your code made, and the observation that came back. That record is the state’s history, and it is what you read when the final answer is wrong. Without it, you have an answer and no way to tell whether the model chose badly or the tool lied.

A common mistake is logging only the final answer and the step count. The symptom appears the first time a run goes wrong: the count says four steps, the answer is empty, and nobody can say which of the four was the mistake. The trace costs a few lines and turns that question into a lookup.

The model proposes, the program disposes

The model never executes anything. It emits a request — a tool name and arguments — and your code decides whether to run it. That separation is the entire safety model, and it is why the tool list is the real attack surface.

It also means the loop is ordinary code that can be tested. A fake model that always asks for the same tool is a perfectly good test of the loop, the budget and the error handling, with no API key involved.

That separation also decides where to put retries. Retrying a model decision is cheap and safe; retrying a tool with an effect is not. Because the loop owns the call, it can apply different policies to each, which is much harder when the model is allowed to call a function directly.

Agents are expensive and unpredictable

Every iteration is at least one model call, and a tool call may be another. A chain costs a predictable amount per request; an agent costs as much as the hardest input needs. That is the trade: flexibility for a bill you cannot forecast exactly.

The mitigation is a budget, not optimism. A maximum number of steps, a maximum number of tool calls and a wall-clock limit turn an unbounded loop into a service with a worst case.

The budget also has a shape. A step limit caps the number of decisions; a token limit caps the cost of each; a wall-clock limit caps the wait. Pick the one that matches the failure you fear, and set it below the point where the run stops being useful. A limit that is never reached in testing is not evidence that it is generous; it is evidence that the test was easy.