EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

callback
A function the library calls at fixed points in a run, such as when a step starts or ends.
chain
A series of steps where each one passes its result to the next, ending in the answer.
invoke
The method that runs a step once with one input and returns its result.
memory
Stored history from earlier turns, added back to a request so the model can refer to what was said.
message
One labelled turn in a chat request, marked as coming from the system, the user or the model.
model
The trained program that reads a message list and returns the next turn.
output parser
The step that turns a model's raw text into a value your program can use, or raises an error.
pipe
The operator that joins two steps into one new step, running the left side first.
prompt
The text a model receives for one request, built from fixed wording and the values filled into it.
prompt template
A reusable prompt with named blanks that are filled in for each request.
runnable
Any step with a standard way to run it, so steps can be joined without knowing each other's code.
streaming
Sending a reply in pieces as it is produced, rather than waiting for the whole text.
tool
A function exposed to a model, with a name, described inputs and a result.
tool call
A model's request to run a named tool with specific arguments, which your code may accept or refuse.

LangChain from the ground up

Messages, not strings

A chat model takes a list of role-tagged messages and returns one. Understanding that list — and that the model remembers nothing between calls — explains most surprising behaviour.

Part 3 of 10

Beginner14 min readBack to Courses

Three roles and one reply

A chat request is a list. system carries the standing instruction, human is a turn from the user, ai is a previous reply from the model. The model reads the whole list and returns a single new ai message. That is the entire protocol.

The list is also the model’s only memory. Nothing is retained between calls: if you want the model to know what it said two turns ago, you send those turns again. A conversation is a growing list that you keep and resend, not a session the model holds.

A two-turn conversation makes the mechanics obvious. After the first reply, you append the assistant message and the new question to the list and send the whole thing again. The model sees its own earlier sentence as an ai message, not as a memory it kept. Remove that message and the follow-up “and the second one?” has nothing to refer to, even though the same model process answered a second earlier.

Statelessness is a feature

Because each call is independent, you can replay it, test it and change it. You can send a different history and get a different answer, which is exactly what makes evaluation possible. A stateful model would make every request depend on everything that came before it on that machine.

It also means the bill grows with the conversation. Every turn resends the earlier ones, so a long chat costs more per message than a short one. That is why trimming and summarising history are part of the design, not an optimisation added later.

Invoking a chat model

from langchain_core.messages import SystemMessage, HumanMessage
from langchain_openai import ChatOpenAI

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

messages = [
    SystemMessage("Answer in one sentence."),
    HumanMessage("What does a vector store do?"),
]

reply = model.invoke(messages)
print(type(reply).__name__)   # AIMessage
print(reply.content)

Render a message list

Run it, then add a third turn where the user says “and the second one?”. Notice that the fake model only sees the list you pass — nothing is remembered for you.

Output
Click "Run" to execute your code

What the model sees is what you sent

When a reply is strange, print the message list rather than the prompt string. A missing system turn, two human turns in a row, or a tool message with no matching call id all look fine in a flattened transcript and wrong in the list. The model has no other view; the list is the entire request.

A common mistake is assuming the framework remembers the system message for you. It does not; you pass it every time. The symptom is an assistant that follows the rules in the first turn and forgets them in the third, because a history-building function dropped the system entry when it trimmed the list.

Temperature is a dial, not a personality

Low temperature makes the model pick the most likely continuation, which is what you want for extraction, classification and anything with a correct answer. Higher temperature samples less likely continuations, which is useful for brainstorming and harmful for parsing.

It does not make a model “creative” in the human sense and it cannot fix a bad prompt. If the instructions are ambiguous, a low temperature gives you the same wrong answer every time, which at least makes the bug reproducible.

Log the exact message list you sent, not just the final prompt string. When a reply is odd, the cause is usually a message that was not what you assumed — a missing system turn, or two human turns in a row.

The reply is an object

A response carries more than text: the role, sometimes tool calls, token usage and a finish reason. Reading .content alone is fine until you need to know whether the model stopped because it finished or because it hit the length limit.

A finish reason of length means the answer was cut off mid-sentence. Treating that as a complete answer is a common source of “the model ignored my instructions” — it did not; it ran out of room.

Read the finish reason before the content. A truncated reply can still parse, still look confident, and still be wrong. The check is one line and it prevents a whole class of confusing downstream failures. The same object also carries token usage, which is the only reliable way to see what a conversation costs before the invoice arrives.

The model remembers nothing

Call `answer` twice with the same question but different histories and compare. Then remove the history argument and see how the second reply loses the context.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

Which role carries the standing instruction?