Legal AI Solution Design MapInterpreting benchmarks for legal work and system design.

Evidence checked through 30 October 2026

What the latest legal AI benchmarks do and do not show

Models can perform individual legal tasks well while still failing to complete a matter. The new evidence sharpens that distinction: 88.56% on broad legal reasoning, 55.77% strict all-pass on tool-using research, and zero fully clean matter runs on LegalReasonBench.

The central result

LegalBench’s 88.56% headline describes performance on mostly short-form reasoning tasks. Legal Research Bench’s 55.77% requires every scored research criterion to pass. On evolving matters, LegalReasonBench reports a 50.8% weighted score but 0% strict all-pass. These measures describe different tasks and must not be presented as one leaderboard.

Harvey’s file-based results show the same gap between local accuracy and completed work: HLAB-Vals-1.1 reports 96.30% criterion pass against 27.08% task pass. Harvey LAB-AA v1.1 reports 95.5% criterion pass but 13.1% hallucination-gated all-pass.

Material updates

LegalBench: high score, narrow task scope

Claude Fable 5 leads at 88.56% across 149 evaluated models. Claude Fable 5.1 is second and Gemini 4 Argon third at 88.30%. This movement within a saturated broad benchmark does not establish that a model can complete legal matters.

Publisher update: 29 September 2026. Keep the site’s headline rounded to 88.6% and retain the warning that this is broad reasoning, not end-to-end work.

LegalReasonBench replaces the current Realm: Legal entry

The publisher’s new benchmark covers 50 expert-authored matters and 122 sequential assignments in a persistent sandbox, with 395 files and 2,236 rubric criteria. The record evolves in 86% of matters, and later work depends on earlier work in 68%.

Gemini 4 Argon leads on weighted rubric score at 50.8%; no model achieved strict all-pass. Argon received newer versions of 17 of the 50 matters, so the publisher’s matching-version analysis treats it and Claude Fable 5.1 (50.6%) as effectively inseparable.

Rename the current record to LegalReasonBench, retain Realm: Legal as an alias, and archive the earlier 2 September numerical snapshot. The current leaderboard is not directly comparable with the preliminary release.

Harvey LAB: keep implementations separate

Vals HLAB-Vals-1.1

Vals removed 17 unsolvable criteria, replaced a mismatched source document, updated its harness and reran the models. The dataset contains 120 held-out tasks; results are not comparable with v1 or with Artificial Analysis’ implementation.

Gemini 4 Argon leads with 96.30% criterion pass and 27.08% strict task pass. Muse Spark Open follows at 25.00%, then Wonder at 23.75%. Record the harness as v1.2.0 where relevant.

Harvey LAB-AA v1.1

This separate Artificial Analysis implementation uses 120 private tasks, three judges and a source-document check that zeroes a task for any material hallucination. Muse Wonder Full leads with 95.5% criterion pass, 13.1% hallucination-gated all-pass and 0.62 material hallucinations per checked task. Do not combine these scores with Vals results.

Artificial Analysis Legal Index: new baseline

The rebuilt index’s leader is Gemini 4 Carbon High at 62, followed by Claude Opus 5.5 at 58 and GPT-6 Astra at 56. It now uses five component benchmarks: AA-Omniscience v1.1, AA-Briefcase v1.1, HLE-Diamond, GDP.pdf and AutomationBench-AA.

Weights are legal knowledge 30%, agentic knowledge work 25%, reasoning 15%, non-hallucination 15%, long context 10% and agentic tool use 5%. This occupationally weighted composite is not a task-completion rate. Treat the October rebuild as a comparability break, not a decline from the earlier 63.

Immigration Bench

Vals’ multi-turn advisory benchmark uses 519 expert-written US immigration questions, with a private 262-question test set and seven user personas. Claude Fable 5.1 leads at 75.44% average rubric-item accuracy, but fully correct answers occur on 26.3% of questions. It informs fact elicitation, current-law retrieval and personalised application; it is not a long-horizon matter or autonomous-agent test.

LexPilot

LexPilot tests whether a model asks for missing facts before advising a lay user. Across 432 authentic Chinese civil judgments and 36 causes of action, downstream claim-level prediction drops by an average of 23.84 points when consultations are underspecified. This supports the map’s “knows when to ask” and missing-facts analysis. Do not invent a model leaderboard headline.

BenGER

BenGER adds German-law coverage through 596 exam-style free-text cases, 531 doctrinal tasks and 12 systems, including a controlled human versus human-AI subset. Its LLM judge was cross-validated against human grading (Pearson r=0.76; Cohen’s κ=0.60). It tests German subsumption reasoning, not tool use or end-to-end matter completion.

GRACE

GRACE evaluates grounded question-reasoning-answer work from Canadian federal legislation across 1,915 instances, including adversarial advocacy, uncertainty and applied reasoning. It is partial evidence for holding a position under challenge and handling missing information. It does not establish live research, current-law validation, sustained advocacy, tool use or end-to-end work; its proof-of-concept compares a fine-tuned 4B model with its base model, not frontier systems.

LexIssue

LexIssue covers Chinese civil litigation with 430 cases and 1,303 expert-annotated disputed issues. It tests free-form issue generation and hierarchical classification from claimant and defendant positions. It does not establish correct rules, legal conclusions, outcomes or completion of litigation. Although the cases are real judgments, claim and defence inputs were reconstructed with GPT-4o, and evaluation uses LLM-assisted alignment.

Vals Legal Index

This revenue-weighted composite combines legal research and drafting across practice areas. Wonder Full leads at 77.86%, followed by Gemini 4 Argon at 76.72% and Gemini 4 Carbon at 75.83%. Keep it in a composite-indices category; 77.86 is not an all-pass completion rate.

Unresolved source caveats

  • PRBench Legal: the narrative’s best hard-subset score remains 0.37, while the live table shows Muse Spark 1.3 at 61.56 ± 1.72 without clarifying whether this is the full set, a revised test or a different scale. Keep “No comparable table recorded” and store the live result as unresolved.
  • ContractScrub: the paper’s prose says all F1 scores are below 0.650, but its table reports 0.655 for Gemini 3.1 Pro. Retain the 75% macro-recall headline, flag the inconsistency and use the table value if F1 is displayed.
  • Artificial Analysis Legal Index: chart and FAQ still disagree on some second- and third-ranked scores. The leader and score of 62 are consistent.
  • RedlineBench and LePhantomCite: no material change in this update. RedlineBench remains led by GPT-5.5 at 50.5%; LePhantomCite by Claude Code with Opus 4.8 at 68.8% F1, while GPT-5 has higher recall at 84.4%.

This page records the supplied November refresh, with publisher evidence dated through 30 October 2026. Scores are presented in their reported units. Comparability breaks and unresolved source conflicts are made explicit rather than merged into a single ranking.

Browse benchmark records Return to overview