242 answers at 0.1 or less. The lawyers said yes to 41.

An answer sheet prints: the 242 answers OpenAI's Decisions API gave at a probability of 0.1 or less. A key sheet comes down over it, and the squares where the benchmark said yes turn red. 41 of the 242.

That is 17 percent of the model's near-certain noes, out of 523 answered legal yes or no questions, wrong by the benchmark's labels.

The averages pass a first look. The mean probability the model gave to a clause labelled absent ran from 0.01 to 0.08 on contract tasks; to a clause labelled present, 0.46 to 0.78.

Underneath, the misses were not spread evenly. On change-of-control clauses the model put 24 of the 50 clauses labelled present below 0.5, and 19 of them at 0.1 or below; one got a probability of 0.0. On anti-assignment clauses, 17 of 50. On non-compete, 8. On consumer contract questions, 1.

Limits. 50 to 100 items per task, so a margin of roughly 7 to 10 points on each. The labels are the benchmark's and I did not audit the misses, so some of the 41 may be the labellers' errors rather than the model's. One run, gpt-6-luna, 8 October, so nothing here says where the same items would land on a second pass.

How many of the 41 would you re-label by hand before deciding whether the model or the key is wrong?

Where the numbers live

• results/metrics.json
• README.md
• results/items.csv
• judge.py

The quiz and the essay; the bin table and per-task means are in results/metrics.json in the repo

Labels are LegalBench's, misses not audited. 50 to 100 items per task. gpt-6-luna, 8 Oct 2026.