Specificity and constraints

Vague is not the same as open

“Write a good summary” gives the model two unknowns: what counts as good, and how long a summary is. It will resolve both from its training distribution, which is the average of everything it has seen — usually longer, more formal and more hedged than you wanted.

A constraint removes one unknown. “At most three sentences, plain text, no bullet points” leaves nothing to infer about shape, so the model spends its capacity on content instead of guessing your preferences.

It helps to picture the model as completing the most ordinary version of your request. Ask for a summary and it produces the summary that appears most often in its training data: a short paragraph, a neutral tone, no numbers unless the source had them. Ask for a summary for an angry customer and the same machinery produces a different average, because the audience is now part of the request. The constraint is not a restriction on the model’s ability; it is a choice about which average you want.

Constraints must be checkable

“Be concise” is a wish. “At most 40 words” is a constraint — a program can count it, and a failing answer can be retried with the count in the message. The same applies to format, allowed labels and required fields.

The useful rule: if you cannot write a function that decides whether the constraint was met, it is not a constraint yet. It is a preference, and preferences belong in examples rather than adjectives.

Constraints in the prompt, checks in the code

PROMPT = """Summarise the ticket in at most 40 words.
Return plain text. Do not name the customer.
If the ticket is not about billing, reply exactly: out of scope."""

def violations(reply: str) -> list[str]:
    problems = []
    if len(reply.split()) > 40:
        problems.append("too long")
    if "\n" in reply.strip():
        problems.append("multiple lines")
    return problems

Write the check first

Before you add a constraint to a prompt, write the function that will check it. “At most 40 words” becomes a word count; “plain text” becomes a check for newlines; “one of three labels” becomes a set membership test. If the check is awkward to write, the constraint is probably a preference, and it belongs in an example instead.

This order also stops a common mistake: a prompt that states a rule the code never verifies. The model follows it most of the time, the failures are invisible, and the first evidence is a customer complaint. The symptom is an evaluation score that looks fine while the output is unusable in a way nobody thought to check.

Too many constraints is its own failure

A prompt with fifteen requirements is a prompt where the model will satisfy twelve and quietly drop three. The dropped ones are not random: they are usually the last written, or the ones that conflict with an earlier rule.

Keep the list short and ordered by importance. If a constraint matters more than the rest, say so explicitly — “the length limit is mandatory; the tone is a preference” tells the model what to sacrifice when it cannot do everything.

A practical ordering is: safety and correctness first, then format, then tone. Put the mandatory ones in a short numbered list and mark the rest as preferences. When the model has to choose, the numbering gives it a rule for choosing, and your checker can tell you which one it sacrificed.

Constraints are how you get consistency

The reason to constrain is not neatness. It is that a constrained output can be compared across runs. Two answers that both obey the same contract can be diffed, scored and regression-tested; two free-form paragraphs can only be read.

That is the bridge to evaluation: every checkable constraint you add to a prompt becomes a check in the harness later.

It also makes the prompt reviewable in a way a free-form instruction is not. A reviewer can see the constraint, the check and the case that exercises it in one place, and can ask whether the three agree. When they disagree, the disagreement is a bug report rather than a matter of taste.