ContractNLI

607 contracts tested against policy-like hypotheses, with evidence spans for each judgement.

The evaluated unit is a model response or component output. Read the source for the exact prompt, tool and harness conditions.

Three-way classification accuracy and evidence identification F1.

Scores remain in the original unit. They are not normalised or combined with results from other benchmarks.

• Primary benchmark source

Tests whether a contract entails, contradicts, or is silent on a policy statement.