Back to Blog ListMostofa Shakib

LLM as a Judge

March 28, 2026•8 min read•Post-Training Research

Using Large Language Models (LLMs) as judges has become a cornerstone of modern AI evaluation. As tasks move to open-ended generation and complex workflows, traditional metrics like BLEU or ROUGE fall short. Instead, we rely on LLMs to score other LLMs. This post explores when to use them, their common biases, structured alignment techniques, and how to write a reliable judge prompt.

When to Use LLM as a Judge

Use LLM as a judge when the task has multiple valid outputs and you care about qualities that are hard to capture with exact matching. Good examples include helpfulness of chatbot responses, quality of explanations, grounded summarization, style controlled generation, open ended QA, and pairwise comparison between two candidate outputs. It is also useful when you need fine grained rubrics. For example, you might want separate scores for factuality, completeness, conciseness, tone, and instruction following. A judge can do that better than reference based metrics.

Use it carefully when the task involves domain expertise and exact truth verification. A judge can say an answer looks reasonable even when it is wrong. This happens often in math, law, medicine, code execution, and grounded factual QA. For these cases, use judge models as one layer, not the final authority. Combine them with deterministic checks, human experts, or tool based verification.

What Makes a Good LLM Judge

  • Capability: Strong understanding of the task, the rubric, and the answer. A weak model will produce shallow verdicts and miss subtle errors.
  • Rubric adherence: Follow the scoring criteria rather than drifting into vague impressions. Without this, scores become inconsistent and hard to interpret.
  • Stability: The same answer should not receive very different scores just because the prompt wording changed or the answer appeared in a different position.
  • Calibration: Correlate well with strong human raters on the desired dimensions.
  • Signal Independence: Do not reward length, confidence, formatting, or similarity to its own style more than actual quality.

It also helps if the judge can explain its decisions in a constrained way because they make error analysis easier and help you detect rubric drift.

What LLM Judges Are Bad At

Judge models struggle with hidden factual errors, subtle logical mistakes, domain specific technical accuracy, and outputs that sound polished but contain false claims. They are especially weak when they cannot independently verify the answer. They also struggle when the rubric is underspecified. If you ask the judge to rate “quality,” it will invent its own definition. That creates noise and inconsistency. Judge models also have difficulty when they must inspect long agent traces with many tool calls and latent state changes. They may miss a critical mistake halfway through the trajectory if the final answer looks good.

Common Judge Biases

1) Position Bias: Position bias happens when the judge tends to favor the first or second answer in a pairwise comparison because of order rather than merit. This is common. Even strong models exhibit it. The fix is straightforward. Randomize answer order and score both orderings when possible. If answer A wins when shown first but loses when shown second, that tells you the comparison is unstable. You can average the outcomes or discard unstable cases for closer review.

2) Verbosity Bias: Verbosity bias means longer answers get higher scores because they look more complete or thoughtful. In reality, they may just contain more filler. This matters a lot because verbosity bias can push your training or model selection in the wrong direction. You end up choosing models that speak more rather than models that answer better. To mitigate it, include brevity or conciseness criteria. Instruct the judge not to reward extra length unless it improves the answer. You can also compare answers after normalizing for length or examining quality per unit of content.

3) Self Enhancement Bias: Self enhancement bias appears when a judge model tends to prefer answers that resemble its own style, reasoning pattern, or common output format. If you judge one model using a very similar model family, this can distort rankings. The mitigation is to use a different model family for judging when possible, or to ensemble multiple judge models. It also helps to strip metadata and blind the judge so it cannot infer the source model easily.

4) Formal Tone Bias: Formal tone bias rewards answers that sound authoritative, even when the underlying content is weak or incorrect. This can lead to over scoring polished but shallow responses. Mitigate it by anchoring evaluation to factual accuracy and reasoning quality rather than tone.

5) Formatting Bias: Formatting bias rewards answers with clean structure, headings, or bullet points regardless of substance. Structure should not outweigh correctness or depth and should only be credited when formatting improves understanding.

6) Confidence Bias: Confidence bias favors assertive answers over uncertain or nuanced ones, even when the confident answer is wrong. This is risky in ambiguous or knowledge limited tasks. Judges should prioritize correctness and justification.

7) Refusal Bias: Refusal bias appears in safety evaluation when judges over reward safe sounding refusals or under penalize subtle harmful compliance. This can skew results toward overly cautious or inconsistent behavior. Mitigation requires safety criteria and distinction between appropriate refusal and unnecessary avoidance.

Aligning LLM Judges

  1. If the rubric is vague, the judge becomes vague. Each criterion should say what good looks like, what bad looks like, and how the score should be assigned.
  2. Give the judge examples of high- and low-quality answers scored by the rubric.
  3. If you do not explicitly warn against verbosity, polished style, unsupported confidence, or decorative formatting, the judge may treat those as quality signals.
  4. Instead of asking for one overall score, ask the judge to evaluate dimensions separately. For example: factual accuracy, completeness, instruction following, conciseness, and safety. This reduces hidden tradeoffs.
  5. Compare the judge’s scores against human judgments on a calibration set. If they do not line up, adjust the rubric, prompt, or model choice.
  6. Swap answer order, vary prompt phrasing, run repeated judgments, and see whether the judge remains stable.

Pointwise vs Pairwise LLM as a Judge

Pointwise judging evaluates one answer at a time against a rubric. This is useful when you need absolute quality scores, pass fail thresholds, or monitoring dashboards. It gives richer per answer diagnostics, but it is harder for the judge because the model must infer an internal scale.

Pairwise judging asks the judge which of two answers is better. This is often easier and more reliable because relative comparison is simpler than assigning absolute scores. It is useful for A/B testing, model ranking, and preference data collection.

Pairwise judgments are better for model comparison when the difference between answers is subtle. Pointwise judgments are often better for longitudinal monitoring and error analysis because they attach scores to individual outputs.

Structured Output for Judge Models

Structured output means the judge returns results in a fixed schema rather than free form prose. For example, winner, correctness_score, factuality_score,completeness_score, failure_categories, and a short rationale.

This matters because free text explanations are hard to aggregate and easy to misread. A schema gives consistency and forces you to define exactly what the judge should output, which sharpens the evaluation design itself.

  • Prompt-level schema: Instruct the judge to return strict JSON with required fields and valid ranges. This helps enforce structure but can drift.
  • Schema-constrained decoding: Use constrained decoding or API-level structured output (e.g., JSON mode) so responses must satisfy a defined schema, reducing malformed outputs.
  • Programmatic validation: Validate responses after generation; checks required fields, numeric ranges, enums, and reject or retry invalid outputs.
  • Deterministic postprocessing: Normalize values, map aliases, handle nulls safely, and version schemas to prevent drift over time.

How to Write a Good Judge Prompt

A strong judge prompt reads less like a casual instruction and more like a contract. You want to remove ambiguity wherever possible. Start by fixing the role: the model is an evaluator applying a rubric, not generating new content. Then define the task context clearly so the judge knows what “good” means in that setting.

The most important part is the rubric. Each dimension should have a concrete definition, not a vague label. For example, “factuality” should explain what counts as supported, unsupported, or contradicted, not just say “be accurate.” Specify what should not influence the decision, such as length, tone, or formatting, unless those are part of the task. Good prompts decompose the evaluation into dimensions before asking for an overall judgment, because that reduces hidden tradeoffs.

Finally, enforce a structured output format and keep it strict. If the model can drift into free text, it will. In practice, stability comes from three things working together: a sharp rubric, calibrated examples, and constrained output.

Sample Judge Prompt Template (Pointwise)

You are an expert evaluator. Your task is to evaluate a candidate answer based
on a defined rubric. Do not generate a new answer. Do not assume facts not present in the provided
context.

### Task
Evaluate the quality of the candidate answer to the given prompt.

### Prompt
{user_prompt}

### Candidate Answer
{candidate_answer}

### (Optional) Reference / Context
{retrieved_context_or_reference}

### Evaluation Rubric

1. Correctness (0–5)
- 5: The answer is fully correct with no errors.
- 3: Mostly correct but contains minor inaccuracies.
- 1: Contains major errors or incorrect conclusions.

2. Factuality / Grounding (0–5)
- 5: All claims are supported by the provided context or are verifiably true.
- 3: Some claims are unsupported or weakly grounded.
- 1: Contains hallucinations or contradicts the context.

3. Completeness (0–5)
- 5: Fully addresses all aspects of the prompt.
- 3: Covers the main idea but misses important details.
- 1: Incomplete or misses key parts.

4. Instruction Following (0–5)
- 5: Fully follows instructions (format, constraints, scope).
- 3: Minor deviations.
- 1: Ignores key instructions.

5. Conciseness (0–5)
- 5: Clear and concise with no unnecessary content.
- 3: Some redundancy or verbosity.
- 1: Excessively verbose or too brief to be useful.

### Important Constraints
- Do NOT reward verbosity or stylistic polish unless it improves clarity.
- Do NOT use outside knowledge unless explicitly allowed.
- Focus on substance over tone.

### Output Format (STRICT JSON ONLY)

{
    "correctness": int,
    "factuality": int,
    "completeness": int,
    "instruction_following": int,
    "conciseness": int,
    "overall_score": float,
    "final_verdict": "pass" | "fail",
    "failure_tags": ["hallucination", "missing_info", "instruction_violation", "verbosity", "other"],
    "rationale": "brief explanation (2-3 sentences)"
}

Sample Judge Prompt Template (Pairwise)

You are an expert evaluator. Your task is to compare two candidate answers and
determine which is better based on the rubric.

### Prompt
{user_prompt}

### Answer A
{answer_a}

### Answer B
{answer_b}

### Evaluation Rubric

Evaluate based on:
- Correctness
- Factuality
- Completeness
- Instruction Following
- Conciseness

### Important Constraints
- Ignore answer order. Position does NOT imply quality.
- Do NOT reward verbosity unless it improves substance.
- Focus on correctness and usefulness over style.

### Output Format (STRICT JSON ONLY)

{
    "winner": "A" | "B" | "tie",
    "reasoning": "brief explanation comparing both answers",
    "dimension_scores": {
        "correctness": {"A": int, "B": int},
        "factuality": {"A": int, "B": int},
        "completeness": {"A": int, "B": int},
        "instruction_following": {"A": int, "B": int},
        "conciseness": {"A": int, "B": int}
    }
}