EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

constraint
A limit on the answer that a program can check, such as a length, an allowed label or a required citation.
context
The background material the model needs to answer well, kept separate from the thing it must act on.
continuation
The text a model writes next, given everything that came before it.
delimited block
Text wrapped by matching markers, such as triple backticks, so its edges are unmistakable.
distribution
The pattern of how often each possible value appears, or how likely each one is.
few-shot example
A worked input and answer pair placed in the prompt so the model can copy the pattern.
instruction
The sentence that names the task, usually starting with a verb.
JSON
A text format for structured data built from names, values, lists and nested braces.
model
The trained program that reads text and predicts what comes next.
output contract
The part of a prompt that states the shape of a valid answer, in words a checker can enforce.
parser
The code that turns the model's text into a value your program can use.
prompt
Everything sent to the model in one request: instructions, background, the input and the answer's required shape.
prompt injection
Text that tries to give the model orders, either typed by a user or hidden inside a document it reads.
register
The level of formality and the kind of wording suited to a particular audience or setting.
schema
A named set of fields with their types and allowed values, used to check an answer's shape.
system message
The opening instruction the model treats as standing policy for the whole exchange.
temperature
A setting that controls how much randomness the model adds when choosing each token.
token
A chunk of text — often part of a word — that the model reads and bills for.
user message
A turn written by the person using the app, treated as data rather than standing policy.

Prompt engineering from the ground up

Few-shot examples

An example teaches a pattern faster than a paragraph of instructions. The risks are the ones a teacher would recognise: the wrong examples, too few of a class, and giving away the answer.

Part 3 of 10

Beginner14 min readBack to Courses

When an example beats a rule

Some tasks are easy to show and hard to describe: a house style, a particular tone, a labelling convention with edge cases. Writing the rule takes three paragraphs and still misses the nuance; two examples carry it immediately.

The reverse is also true. If the task has one clear rule — “reply with the sentiment label only” — examples add tokens and can introduce a pattern you did not intend.

The mechanism is pattern completion. The examples are tokens in the same context as the new input, so the model conditions on them exactly as it conditions on the instruction. It does not extract a rule and apply it; it continues the sequence. That is why a single inconsistent example can outweigh a sentence of guidance, and why the last example before the input has the most influence.

The model copies everything

An example is not just content; it is a template. If every example answer begins with “Sure!”, the model will begin with “Sure!”. If every example is positive, a neutral input may be labelled positive because that is the pattern in front of it.

That means examples have to be representative and balanced. Two examples of the same class teach the model that the class is common, which is a claim about your data you probably did not mean to make.

A few-shot prompt

EXAMPLES = [
    ("The delivery never arrived.", "complaint"),
    ("How do I change my address?", "question"),
    ("Thanks, that fixed it!", "praise"),
]

prompt = "Classify each message as complaint, question or praise.\n\n"
for text, label in EXAMPLES:
    prompt += f"Message: {text}\nLabel: {label}\n\n"
prompt += f"Message: {incoming}\nLabel:"

Build the prompt from examples

Run the builder, then add a fourth example whose label is “question”. Confirm the prompt still ends with an open “Label:” for the model to complete.

Output
Click "Run" to execute your code

The order and the ending matter

Examples are read as a sequence, and the last one before the input has the most influence. Put the example closest to the test item in shape and label, and end with the token you want the model to produce. A classification prompt should finish with “Label:” and nothing after it; an extraction prompt should finish with the opening brace of the object.

A common mistake is adding explanations to the examples. “The delivery never arrived, so this is a complaint because the customer is unhappy” teaches the model to write explanations, and the next reply includes one even though the instruction asked for a bare label. Keep the examples in exactly the format you want back, including the silence after the final cue.

Choosing examples at runtime

A fixed set of examples is fine for a stable task. When the inputs vary widely, picking the examples most similar to the current input helps: a question about refunds is better taught by refund examples than by a random sample.

Similarity is usually lexical or embedding-based, and it has the same failure as retrieval — it can pick examples that are similar in wording and different in the thing being labelled. Check the selection on a labelled set before trusting it.

A concrete failure: the query “when will my refund arrive” retrieves the complaint “refund not received after ten days” because the words overlap, and the model copies the complaint label for a question. The fix is not a better similarity metric alone; it is to keep the label balanced across the retrieved set and to check the selection against a labelled sample.

Never include the test case itself as an example. It looks like a spectacular accuracy improvement and measures nothing.

How many is enough

Two or three well-chosen examples cover most formatting tasks. Ten are useful when the label set is large or the edge cases are subtle. Beyond that, the prompt grows faster than the accuracy, and every example is paid for on every call.

The number to tune is not “how many can fit” but “how many until the failures stop being about format and start being about judgement”. Those need different fixes.

One more signal: count the examples by label before you ship. If the counts are uneven, either add examples until they are not, or write down why the imbalance reflects the traffic. An imbalance nobody wrote down is an accident, and the model will treat it as a fact about the world.

Pick the nearest examples

The selector chooses examples by word overlap. Run it for two inputs and see how the chosen examples change; then add an example that is lexically close but labelled differently.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

When is a few-shot example better than an instruction?