LLM as judge

What a judge is good for

For open-ended answers there is no reference string to compare against. A judge reads the question, the answer and a rubric, and returns a verdict with a reason. It scales to thousands of cases and costs a fraction of human review.

It is a classifier built from a prompt. Like any classifier it needs a definition of correctness, examples of each label, and a test of whether it agrees with a person on cases where the answer is known.

A worked rubric has three clauses: states the five-day window, cites a source, adds no claim absent from the context. Each is binary, so the verdict is a list of failed clauses rather than a score out of ten. “Failed with reason: missing citation” is actionable; “7 out of 10” is not, because it does not say which of the three things to fix.

Pointwise or pairwise

Pointwise asks “is this answer good?” on a scale. It is cheap and gives an absolute score, and the scale drifts: a judge that gives threes today may give fours tomorrow after a prompt change.

Pairwise asks “which of these two answers is better?”. It is more stable and better at fine distinctions, and it cannot tell you whether either answer is good. Most evaluation systems use pointwise for pass/fail and pairwise for choosing between variants.

Scale drift is easy to demonstrate. Run the same fifty answers today and next week with an unchanged prompt, and the average moves by half a point because the model was updated or the temperature was not pinned. Pairwise comparison is immune to that drift, because it never asks for an absolute number.

A judge with a fixed shape

from pydantic import BaseModel, Field

class Verdict(BaseModel):
    passed: bool = Field(description="true only if every rubric item is met")
    reason: str = Field(description="one sentence naming the first failed item")

judge = prompt | model.with_structured_output(Verdict)

verdict = judge.invoke({
    "question": question,
    "answer": answer,
    "rubric": "1. States the five-day window. 2. Cites a source. 3. Adds no new claims.",
})
print(verdict.passed, verdict.reason)

The biases are known

Judges prefer longer answers, answers that look confident, and the first option in a pairwise comparison. They are also generous when the rubric is vague and harsh when it is a checklist of many items.

The standard defences are mechanical: swap the order in pairwise comparisons and count a win only if it survives both orders; state the length limit in the rubric; keep the rubric to a few checkable clauses.

Length bias is the most reliable of them. Give the judge two answers with the same facts and make one twice as long; the longer one wins far more often than chance. The defence is in the rubric: state a length limit, ask for the shortest answer that contains the facts, and score verbosity as a failure rather than a virtue.

Write the rubric as a checklist

A rubric that says “be accurate and helpful” measures nothing, because every answer can be argued into it. A checklist says what must be present and what must not: the window, the citation, no new claims, no competitor names. Each item is a yes or no that two people can apply to the same answer.

Keep the list short. Ten clauses turn the judge into a literalist that fails good answers for a missing comma; three or four cover most of what matters, and the rest belong in a separate review pass.

Calibrate against people

Label fifty cases by hand. Run the judge on the same fifty. Compute agreement, and read the disagreements. If the judge is stricter than you, the rubric is too literal; if it is more lenient, the rubric is missing a clause.

The number to report is not the judge’s score but the judge’s agreement with the reference labels. Without that, a judge is an opinion with an API key.

Agreement is the number to report: forty of fifty, or 0.8. Read the ten disagreements before trusting the other forty, because they show whether the judge is stricter, looser or simply measuring a different question. If agreement is high only on easy cases, report it per slice rather than as one average.