Capstone: a bounded research agent

The shape of the finished loop

The agent takes a question, searches, reads what it found, and answers with a citation. The loop stops on an answer, on a budget, or when an observation repeats. A permission check runs before every tool, and a refund-style action waits for approval.

That is the same skeleton as the first chapter, with the parts the course added: schemas, a plan, a progress check, a policy and a trace. Nothing in it needs a model to run, which is what makes it testable.

Walk one run: the model asks to search for “refund”, the policy allows it, the tool returns two document ids, the model asks to read the first, the observation is new, the model answers with the sentence it read. Five trace lines, two tool calls, one answer. If the answer is wrong, the trace shows whether the search returned the wrong document, the read returned nothing, or the model ignored what it read.

The trace is the product

Every step writes a line: the action, the arguments, the decision, the observation. When the agent is wrong, the trace says whether it chose badly, was refused, or read something misleading.

That record is also what makes an agent reviewable. A reviewer can read ten lines and see why the answer exists, which is more than a chat transcript usually offers.

The trace should be append-only and written before the effect, not after it. A line that says “about to issue refund” is what lets you reconstruct a crash; a line written after the call is missing exactly when the process died. That ordering costs nothing and turns an ambiguous failure into a sequence you can read.

The same loop with a framework

agent = create_react_agent(model, tools=[search, read], checkpointer=memory)
result = agent.invoke(
    {"messages": [("user", question)]},
    config={"configurable": {"thread_id": "run-1"}, "recursion_limit": 8},
)
print(result["messages"][-1].content)

Test the exits, not the answer

Three tests cover most of the loop: a scripted model that answers immediately, one that repeats the same observation, and one that never stops. The first proves the happy path, the second proves the progress check, the third proves the budget. None needs a network, and all three run in milliseconds.

A common mistake is testing only with a live model and a question it happens to answer well. That test passes for the wrong reasons and fails when the provider changes a default. The symptom is a regression that appears in production with no code change. Keep the scripted runs as fixtures; they are the only tests that tell you the loop itself still works.

What the capstone leaves out

A real system adds streaming, so the user sees progress; persistence, so a run survives a restart; and evaluation, so a prompt change can be measured. Each is an addition to a part that already exists, not a redesign.

The one thing not to leave out is the budget. An agent without a limit is a demo that works until the first adversarial input.

Add those in the order the product needs them, not the order the framework documents them. Streaming changes the user experience first; persistence changes what survives a restart; evaluation changes what you dare to change. Each one attaches to a seam that already exists in the loop. A capstone is a starting point, not a finished product, and the seams you leave visible are the ones the next person can extend.

Where to go next

The LangGraph course takes this loop and makes it a state machine with checkpoints, interrupts and resumable threads. The Evaluation course is how you tell whether a change to the prompt made the agent better.

The RAG course is the retrieval half in depth: chunking, ranking and grounding, which is what the search tool in this capstone is pretending to be.