The Ultimate Guide to LLM-as-a-Judge

Last updated: July 10, 2026.

Most articles about LLM-as-a-judge are either academic surveys or toy examples. This one is different: it's the real judge prompt behind PromptLayer's AI Judge eval column.

Copy it, adapt it, and give us feedback!

The prompt (v18, running in production today)

System message (model: GPT-5):

You are an AI system designed to evaluate boolean assertions about provided data.
CRITICAL INSTRUCTIONS:
1. Respond with exactly "true" or "false" based on the assertion and data
2. When the assertion states something AS A FACT (e.g., "Your name is X",
   "You are Y"), accept it as true unless the data explicitly contradicts it
3. Focus ONLY on what the data shows, not on what you can or cannot verify
4. The assertion will always be about the data. The assertion may use other
   terms like "Output", "Answer", "Response" or "Value". But it will always
   be an assertion on the data.
5. If evidence is partial but supportive, lean toward "true"
6. Only respond "false" when:
   - The data explicitly contradicts the assertion
   - The assertion makes a claim that is demonstrably absent from the data
   - The assertion requires something that clearly did not occur
DO NOT:
- Add meta-reasoning about verification capabilities
- Default to "false" just because evidence is incomplete
- Question assertions that are stated as facts about identity or role

User message:

# Assertion:
{prompt}

# Data
{data}

Why it looks like this: the design decisions

• Binary verdicts, not 1–10 scores. Score-based judges feel more informative but are dramatically less reliable — models can't hold a stable definition of "7 vs 8" across calls, and score drift poisons regression comparisons. A batch of specific true/false assertions gives you more signal than one vague rubric score, and it's diffable across prompt versions.
• The negative instructions are scar tissue. Every line in the DO NOT list corresponds to a real failure mode we observed at scale. We found that a lot of the frustration with our users was when their assertions were ambiguous and the model chose the wrong direction in the case of ambiguity. More on this below.
• Reasoning and citation in the output schema, not the verdict text. The verdict must be machine-parseable ("exactly true or false"); the explanation lives in a separate structured field so customers can audit why without breaking the parser.

18 versions of getting it wrong, then less wrong

Here's the actual version history from our Prompt Registry, with the eras that mattered:

v1 — the naive one-liner (gpt-4o)

You are an assertion manager. Your job is to asses the user message
and respond with true or false only

103 characters, typo included. It mostly worked in demos and failed in production in every way a judge can fail: it hallucinated judgments about data it invented, answered questions instead of evaluating assertions, and had no consistent policy for ambiguity.

v5–v8 — the hedging era ("adding more hints")

Re-read the data several times and re-read the assertion the user
provides. Please make sure you only answer based on the data and
do not make anything up

Classic first response to hallucination: beg the model to be careful. It helped at the margins. These versions also downgraded from gpt-4o to gpt-4o-mini — a judge runs on every eval cell, so cost per verdict matters, and a well-prompted small model beat a lazily-prompted big one.

v9 — the great simplification: shorter, and pick a default

Based on the user query, respond with "true" or "false".
If you aren't sure, give it "false".

The 418-character hedging prompt became 179 characters, and — crucially — the judge got an explicit ambiguity policy: default to false. Precision over recall. This felt principled: a strict judge doesn't let broken outputs sneak through. Remember this decision; it comes back to bite.

v10–v13 — structure experiments

Four versions of moving furniture, each testable against the same dataset: put the data in its own message vs. the system prompt (v10), add a reasoning key to the output schema (v11), move the assertion into the system message and back (v12), require a citation pointing at the evidence (v13). The reasoning key and citation survived; forcing everything into the system message didn't. If you're not versioning your judge prompt, this stage is where you lose track of what you actually tested.

v14–v18 — the model upgrade trap

When GPT-5 shipped, we swapped it in (v14 tested gpt-5-mini, v15 moved to gpt-5) expecting free accuracy. Instead we got a brand-new failure mode: the smarter model started adding meta-reasoning about its own epistemics. Asked to verify "Your name is Alex" against a transcript, it would answer false — reasoning that it couldn't truly verify anyone's name. Combined with v9's "when unsure, say false" policy, the judge became systematically over-skeptical: false negatives everywhere, customers seeing legitimate outputs flagged as failures.

v17 and v18 are the correction — the prompt grew from 189 to ~1,050 characters, almost all of it calibration rules: accept stated facts unless contradicted, lean true on partial-but-supportive evidence, and the DO NOT list that bans verification meta-reasoning outright.

The lesson we'd tattoo on every eval pipeline: a judge prompt is calibrated to a model, not just to a task. A model upgrade silently re-biases your judge — our accuracy issue wasn't a prompt regression or a data change, it was GPT-5 being differently smart. If your eval provider (or your team) swaps judge models without re-calibrating, every historical eval score changes meaning.

Judging the judge

How did we know the judge was drifting? Customers told us — verdicts that "added logic to the assertion, or just missed the assertion completely." That kicked off the calibration work that still runs today.

Bootstrapping ground truth from production history

We didn't have a labeled dataset, so we built one from what we did have: historic production requests. We pulled 1,000 real assertion calls from our request logs, re-ran them through several more powerful models, and used the agreement (plus review of the disagreements) to establish a ground-truth verdict for every row. Total hand-labeling from scratch: zero. This is the single most repeatable trick in this post — your production logs are an unlabeled eval dataset waiting to happen.

Then we ran every judge candidate against those 1,000 rows in one evaluation table. Results from the actual run (August 2025):

Note what happened: the shiny candidates lost. The small-model variants that looked great on cost would have been a double-digit accuracy regression, and even our "improved" prompt underperformed the incumbent. Without the eval we would have shipped at least one of these on vibes. With it, we kept iterating until the v17/v18 rewrite actually beat the baseline — and every number in that table is one eval-table screenshot away from being verified.

The judge that judges the judge

We also built a second prompt, llm-assertion-checker, whose only job is auditing the judge: it takes the data, the assertion, and the judge's verdict, and evaluates whether the judge did its job. We ran it across complaint cases to find the failure patterns that became v18's DO NOT list.

We also maintain per-model variants (llm-assertion-gpt-suggestion, -claude-suggestion, -gemini-suggestion) for testing judge behavior across providers — because the calibration lesson above cuts both ways: the same prompt is a different judge on a different model.

Your version of this doesn't need to be fancy: keep a labeled set of (data, assertion, correct verdict) triples — especially ones your judge got wrong — and run every judge-prompt change and every judge-model change against it before promoting. In PromptLayer this is a regression dataset attached to the prompt, so a new version of the judge can't reach production without passing the cases that burned us before. (Our eval fundamentals guide covers building these datasets.)

Can you trust an LLM judge? Shrink the surface area for bias

The honest answer to "how can you trust a model to grade another model?" isn't "the judge is unbiased" — no LLM is. It's that you control how much room the judge has to be biased, and that room is a direct function of how specific your question is.

Think of every judgment as having a surface area for error. A generic question — "Is this a good response?" — has enormous surface area: the model has to silently decide what "good" means, and it fills that vacuum with its own priors (verbosity bias, tone preferences, a weakness for confident-sounding answers). That is exactly where hallucination and bias live: in the gap between a vague question and a single defensible answer.

A specific boolean assertion collapses that surface area. "The response includes a phone number in E.164 format" or "The answer cites at least one of the provided sources" has a near-deterministic answer — there is almost nowhere for the model to inject an opinion. You've turned a judgment call into a lookup. This is the whole reason our design leans on many narrow true/false assertions instead of one broad rubric score: each assertion is small enough that the model behaves like a checker, not a critic.

So the practical recipe for a trustworthy judge is to decompose vague quality into the most specific, verifiable booleans you can write. Instead of "Is this response helpful?", ask: does it answer the question that was asked? does it stay grounded in the provided context? does it avoid inventing facts absent from the data? Each of those is checkable; the vague original is not. The more specific the assertion, the smaller the area of hallucination — and the more a disagreement with your ground-truth set points at a real bug instead of a coin flip.

This also makes trust measurable rather than a matter of faith. Because each assertion is narrow, when the judge disagrees with a human label you can usually read the row and see exactly why — the assertion was ambiguous, or the data genuinely was borderline. That's the loop from the calibration section above: specific questions don't just reduce bias, they make whatever bias remains diagnosable.

LLM-as-a-judge best practices, distilled from 18 versions

1. Prefer many binary assertions over one rubric score. Reliability beats granularity.
2. Give the judge an explicit ambiguity policy — and choose it consciously. "Default false" biases toward false alarms; "lean true on partial evidence" biases toward misses. The right choice depends on what a wrong verdict costs you. Just don't leave it to the model's mood.
3. Write negative instructions from observed failures, not imagination. Our DO NOT list is short because each line earned its place.
4. Separate the verdict channel from the explanation channel. Parseable verdict, auditable reasoning, citation to evidence.
5. Re-calibrate on every judge-model change. Treat a model swap like a prompt rewrite, because functionally it is one.
6. Version the judge like production code. The 18-version history above only exists because the judge lives in our Prompt Registry with diffs and rollback — when eval scores shift, we can answer "did the judge change or did the app change?" in one click.
7. Keep a judge-regression dataset. Every complaint becomes a test case; the judge never fails the same way twice.

When a judge is the wrong tool

LLM-as-a-judge is one leg of a three-legged stool. Use code checks (exact match, regex, JSON schema validation) when correctness is objective — they're free, instant, and never hallucinate. Use human review to establish ground truth, calibrate your judges, and evaluate genuinely subjective quality — a judge that's never been checked against human labels is an unvalidated instrument. Use the judge for scaled evaluation of semantic properties: tone, groundedness, instruction-following, assertion checking. In PromptLayer, all three run side-by-side as columns in the same evaluation table, which is exactly how the three legs are meant to work: humans set the standard, code enforces the objective parts, judges scale the judgment calls.

What's next: v19 might not be GPT-5 at all

The same bootstrapped dataset that guards against regressions also unlocks the next move: we recently fine-tuned an open-source Gemma 4B model on our assertion data and matched GPT-5's judge accuracy at roughly 1/10th the cost and 1/4th the latency. A judge runs on every eval cell, so that's not a rounding error — it changes what "evaluate everything" costs. When it takes over as the production assertion judge, this post gets its v19 update.

We're walking through exactly how we did it — the dataset, the fine-tune, and the eval that proved parity — in a live session: register for the webinar here.

FAQ

What is LLM-as-a-judge?

LLM-as-a-judge uses a language model to evaluate the outputs of another model (or the same model) against criteria you define — factual grounding, tone, instruction-following, or specific boolean assertions. It scales far beyond human review and handles subjective qualities that string matching can't, at the cost of needing calibration and monitoring like any measurement instrument.

LLM-as-a-judge vs. human evaluation — which should I use?

Both, in a specific relationship: humans define ground truth on a sample, the judge is validated against that sample, then the judge scales to the volume humans can't cover. A judge that disagrees with your human labels more than ~10% of the time isn't ready to run unsupervised. Spot-check continuously — our checker-prompt pattern above is one way to automate that.

Which model should I use as the judge?

The one you've calibrated. Bigger models aren't automatically better judges — our GPT-5 upgrade initially made the judge worse because its extra reasoning manifested as epistemic hedging. Smaller models with well-tuned prompts are often the right cost/accuracy tradeoff for high-volume eval columns; save the frontier model for low-volume, high-nuance judgments.

How do I stop the judge from hallucinating or drifting?

Constrain the output format, require citations to the provided data, enumerate the conditions for each verdict, ban meta-reasoning explicitly, and — the part most teams skip — maintain a labeled regression set and rerun it on every prompt or model change. Drift you don't measure is drift you ship.

Is LLM-as-a-judge biased?

It can be, but you control how much. Bias creeps in through vague questions: ask "Is this a good response?" and the model fills the gap with its own priors — verbosity bias, tone preferences, a pull toward confident-sounding answers. The fix is to shrink the surface area for bias by asking the most specific, verifiable boolean questions you can. "The answer cites at least one provided source" has a near-deterministic answer; "is this helpful?" does not. Many narrow true/false assertions leave far less room for hallucination than one broad rubric score, and they make any remaining bias visible when you check the judge against human-labeled ground truth.

The LLM Assertion column ships in every PromptLayer evaluation — alongside code checks, human grading, and (soon) evals you can run from Python and CI. Start free, or read the eval scoring docs.