EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

constraint
A limit on the answer that a program can check, such as a length, an allowed label or a required citation.
context
The background material the model needs to answer well, kept separate from the thing it must act on.
continuation
The text a model writes next, given everything that came before it.
delimited block
Text wrapped by matching markers, such as triple backticks, so its edges are unmistakable.
distribution
The pattern of how often each possible value appears, or how likely each one is.
few-shot example
A worked input and answer pair placed in the prompt so the model can copy the pattern.
instruction
The sentence that names the task, usually starting with a verb.
JSON
A text format for structured data built from names, values, lists and nested braces.
model
The trained program that reads text and predicts what comes next.
output contract
The part of a prompt that states the shape of a valid answer, in words a checker can enforce.
parser
The code that turns the model's text into a value your program can use.
prompt
Everything sent to the model in one request: instructions, background, the input and the answer's required shape.
prompt injection
Text that tries to give the model orders, either typed by a user or hidden inside a document it reads.
register
The level of formality and the kind of wording suited to a particular audience or setting.
schema
A named set of fields with their types and allowed values, used to check an answer's shape.
system message
The opening instruction the model treats as standing policy for the whole exchange.
temperature
A setting that controls how much randomness the model adds when choosing each token.
token
A chunk of text — often part of a word — that the model reads and bills for.
user message
A turn written by the person using the app, treated as data rather than standing policy.

Prompt engineering from the ground up

Reasoning and decomposition

Asking for the steps helps on multi-step problems and hurts on simple ones. The reliable version is to decompose the task in code, not to hope the model narrates.

Part 5 of 10

Intermediate15 min readBack to Courses

What “think step by step” actually buys

A model generates one token at a time, so a long answer gives it more room to compute. On arithmetic, date reasoning and multi-hop questions, asking for intermediate steps measurably improves accuracy: the intermediate tokens are working memory.

On a single-label classification it does nothing except cost tokens and latency. Worse, the narration can introduce a plausible-sounding reason that contradicts the label, and a later step may follow the reason instead of the answer.

A worked example makes the difference concrete. Ask “which plan is cheaper for seven seats” and a single call has to find the price list, multiply, compare and phrase the answer in one generation. Split it into extract, lookup and compare, and each call has one job; the multiplication happens in code, where it cannot be approximated. The narration was never the useful part — the intermediate values were.

Decompose in code where you can

The most reliable decomposition is not a prompt technique. It is splitting the job into separate calls with typed inputs and outputs: extract the entities, then look up each one, then compose the answer. Each step can be tested, retried and replaced.

A model asked to do all three at once has to keep everything in one generation. A pipeline does not, and its failures are localised to the step that produced them.

Two shapes, one task

# One call: the model must hold everything at once.
answer = model.invoke("Which plan is cheaper for 7 seats, Starter or Team?")

# Three calls: each one is small and checkable.
seats = extract_seats(question)          # 7
prices = lookup_prices(seats)            # {"starter": 0, "team": 63}
answer = compare(prices)                 # deterministic

Break the question into steps

Run the pipeline, then add a third plan to the price table. Notice that only the lookup changes — the extraction and the comparison stay the same.

Output
Click "Run" to execute your code

Decomposition is a data design

Splitting a task into calls only helps if the values between the calls are typed and checkable. “Extract the seat count” should return an integer, not a sentence containing one; “look up the price” should return a number, not a paragraph. Once the intermediate values are data, the comparison at the end can be ordinary code and the model is used only where judgement is needed.

A common mistake is decomposing the prompt but not the program: three paragraphs in one message asking the model to first extract, then calculate, then answer. The failure rate barely moves, because the model is still holding everything in one generation. The symptom is that errors persist after “breaking it down”, which is the signal that the breakdown happened in prose rather than in code.

Reasoning you cannot see is reasoning you cannot check

A narrated chain is not evidence. Models produce explanations that are coherent and post-hoc; the stated reason may not be the one that produced the answer. Treat the narration as a draft, not a proof.

That is why the useful pattern is a short answer plus a checkable artefact: the extracted number, the cited passage, the SQL query. Those can be verified by code, which is stronger than reading a paragraph of self-report.

What to log instead: the extracted value, the lookup result, the final comparison, each with the input that produced it. Those are small, checkable and safe to keep. The narration is none of those things, which is why it belongs in a debug view at most and never in the answer.

Do not put hidden chain-of-thought into logs or prompts you will show a user. It is verbose, it can leak the prompt, and it is not the explanation you want to defend.

Self-consistency, and its price

When a task is genuinely ambiguous, sampling several answers and taking the majority often beats one careful answer. It trades money for accuracy, and it only works when the answer is short enough to compare.

For anything with a deterministic checker — a sum, a schema, a unit test — checking is better than voting. Use sampling when correctness cannot be computed, and a checker when it can.

Self-consistency also has a failure mode worth naming: the samples can agree on the same wrong answer. Majority voting measures agreement, not correctness, and a model with a systematic misconception will produce it consistently. If three samples agree, that is evidence of stability, not proof; keep a checker or a labelled case nearby.

Vote, or check

Three samples disagree. Take the majority, then replace the vote with an arithmetic check and see which answer survives.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

When does asking for steps help?