Issue No. 1 · Page V · June 2026
Methodology & Colophon
How this issue was built, what the numbers mean, what they do not mean, and where to go if you want to check the work yourself.
What we measured and how
The unit of this issue is a battery of three bar-exam-grade essays on fact patterns authored privately and run for the first time in April 2026. Why that matters — and why it separates this result from a leaderboard number — is argued fully on Page I.
Twelve models were given the same battery, in the same harness, under the
same conditions. Each received the same task description, no prior context,
and no coaching. The same point-sheet rubric was applied to every response.
The same judge — claude-opus-4-6 — evaluated every
answer. The harness wrote results into a database the moment each model
produced them; judgment followed on the same artifact record.
These three essays were chosen because they are dense, interdependent, and trap-laden — the kind of problem where a model that pattern-matches instead of reasoning will fail quietly and confidently. The framing of what that means is on Page I.
The battery
A testamentary conveyance through a common-law jurisdiction that has never abolished the fee tail, retains the classical wait-and-see-free Rule Against Perpetuities, applies the fertile octogenarian and unborn-widow presumptions, and construes class gifts under the rule of convenience. The model was required to trace a multi-generation estate chain and produce a hand-generated SVG diagram showing the chain of title with each estate's validity annotated. This is the essay with the 13-item point-sheet (items a through m) used as anchors for the holistic 1–5 memo score. The diagram is scored separately by a visual judge, also 1–5.
A multi-issue fact pattern requiring analysis of Fourth and Fifth Amendment violations, including a warrantless-search issue, a custodial-interrogation issue, and a Monell-liability question under 42 U.S.C. §1983. The model was required to identify the doctrinal elements, apply them to the facts, and reach a holding on each. The rubric rewards correct classification, correct application, and correctly reaching the operative conclusion.
A multi-statement evidence problem requiring classification of each statement as hearsay or non-hearsay, identification of applicable exceptions (including a homicide-only exception that traps the over-cautious reader), and analysis of Confrontation Clause implications under Crawford v. Washington. Several items are designed to surface the difference between classification failure and conclusion failure — a distinction that matters when comparing models across capability levels.
The rubric: Property memo
Each essay was scored holistically on a 1–5 scale anchored against a point-sheet rubric. The Property memo uses thirteen analytical items (a–m) as the anchoring framework: a score of 5 means the memo addresses items a through l correctly with clean IRAC structure (issue, rule, application, conclusion — the standard legal-memo format); item m is desirable but not required for a top score. The diagram is scored separately by a visual judge, also on a 1–5 scale. A model that gets the law right on every analytical node but draws the chain wrong receives a separate diagram penalty — the scoring surface keeps those apart.
| Item | Issue the memo must address |
|---|---|
| a | Fee tail classification for Arthur’s present interest. |
| b | Character of interests following a fee tail as remainders (not executory interests). |
| c | Widow’s interest as a contingent remainder for life — contingent because the widow is unascertainable until Arthur’s death (the “unborn widow” trap). |
| d | Grandchildren’s interest as a contingent remainder to an open class, subject to the condition precedent of reaching 25. |
| e | Society’s interest as an alternative contingent remainder. |
| f | Identification of measuring lives (Arthur, Bernice, Celia, Dmitri, Octavia, Society trustees). |
| g | RAP analysis of the widow’s life estate: VALID (vests, if at all, at Arthur’s death). |
| h | RAP analysis of the grandchildren’s remainder: VOID — afterborn-child + afterborn-grandchild mechanism; all-or-nothing rule (Leake v. Robinson) voids the entire class gift. |
| i | RAP analysis of the Society’s interest: VOID (takes only on the same remote event; charity-to-non-charity gifts ARE subject to RAP). |
| j | Fertile-octogenarian presumption is IRREBUTTABLE at common law — Octavia’s medical evidence is irrelevant. |
| k | Octavia is not a measuring life for these interests. |
| l | Resulting state of title: Arthur holds the fee tail; widow’s contingent life estate stands; everything after fails and reverts to Helena’s estate, passing under residuary to Arthur. |
| m | Modern-statute footnote (fee tail abolished in most states; USRAP wait-and-see would save the voided interests). Optional — desirable but not required for a score of 5. |
| Memo scored 1–5 holistically against the above point-sheet. Diagram scored 1–5 separately by visual judge. Property total contributes up to 10 toward the campaign maximum of 20 (Criminal Procedure memo /5 + Evidence memo /5 = remaining 10). | |
The Criminal Procedure and Evidence essays are each scored 1–5 on their respective memo rubrics. All three memos plus the Property diagram combine for a campaign maximum of 20 points: Property memo (/5) + Property diagram (/5) + Criminal Procedure memo (/5) + Evidence memo (/5).
Channel correction
All costs in this issue are corrected to a common channel so that price comparisons are meaningful. Without correction, a model run on its native API would appear to cost differently from the same model run through an intermediary — not because the compute differed, but because the billing path did. The correction applies published per-token rates to observed token counts and normalizes to a single channel baseline.
Anthropic models were scored on their native Claude Code harness, which
charges at the published direct-API rate. OpenAI and Google models were
scored via OpenRouter and are marked below; a direct-API re-run is planned
for a future issue. ‡ xAI
(grok) and DeepSeek were also scored via OpenRouter by design
— the lab holds no direct keys for those providers.
Three of the twelve cells are anchors drawn from the predecessor
EV-003 law-essays campaign, which ran the same tasks against
the same rubric with the same judge. They are included as a
cross-campaign consistency check: if the rubric and judge are stable, the
anchor cells should produce stable scores.
| Model | Channel |
|---|---|
| claude-opus-4.8 | native |
| claude-fable-5 | native |
| claude-opus-4.7 | native |
| claude-opus-4.6 | native |
| haiku-4.5 | native |
| gpt-5.5‡ | OpenRouter |
| gpt-5.4‡ | OpenRouter |
| gemini-3.5-flash‡ | OpenRouter |
| gemini-3.1-pro‡ | OpenRouter |
| grok-4.3 | OpenRouter |
| deepseek-v4-pro | OpenRouter |
| deepseek-v3.2 | OpenRouter |
‡ OpenAI and Google models were scored via OpenRouter. Costs have been corrected to published direct-API rates for comparison purposes. A direct-API re-run is planned for a future campaign; results may differ slightly at the margin.
Why we show the work
GunnerBench publishes the actual prompts and the models' actual answers so a reader can judge the reasoning rather than trusting a number. That is not a policy choice or a positioning posture — it is what this kind of measurement is for.
A score is a claim. The work is the evidence. The exhibits in this issue are the actual memos and diagrams from the Property problem — all twelve models’ answers, with an excerpt of the prompt, shown as-received. A reader who works through two or three exhibits will come away with a felt sense of what a model actually does with a hard interdependent problem, which is what they need to make a trust-allocation decision. That felt sense is not computable from the number alone.
This is also why the issue foregrounds the failure modes alongside the ceiling scores. The model that wrote a polished, well-drafted analysis and got the law exactly backwards — scoring a perfect five for drawing the wrong conclusion crisply — is the most important exhibit in this issue. A confident-wrong answer, well-drafted and visually polished, is the one a busy reviewer is most likely to wave through. Seeing it is not optional if you are responsible for work that travels forward from it.
A caveat in fairness to that exhibit: the model’s misstep — that the remainders following Arthur’s fee tail escape the Rule Against Perpetuities — is the strongest classical argument on the table, since a barrable fee tail leaves those remainders destructible, and destructible interests are traditionally exempt from the Rule. Our point-sheet scores it against the model, on the view that Franklin’s retained fee tail does not save the gifts over on these facts; reasonable authorities differ, and we scored to the rubric rather than around it. That a model can invoke a recognized doctrine, draft it cleanly, and still land where the answer key marks it wrong is not a softer version of the lesson — it is the lesson.
Honesty rails
Every campaign produces a finding. Not every finding generalizes as far as the headline implies. These are the limits we hold ourselves to when interpreting Issue No. 1.
What this issue does and does not establish
- “Commoditized / saturated” rests on this three-essay battery. That is a strong directional signal on dense, interdependent multi-chain reasoning problems. It is not a universal law. Breadth — more doctrines, more task types, more domains — is the bulletproofing follow-up.
- N = 1 per model. A single run has a real noise floor. Rank coarsely within the top cluster; the difference between 19 and 20 on one run is smaller than the difference between 15 and 19.
-
The judge is
claude-opus-4-6— an Anthropic model judging a field that includes Anthropic models. The rubric is point-sheet style rather than holistic, which partially insulates it from judge-model bias, but that insulation is imperfect. The lab has examined self-preference on this battery and will treat the question more fully in a future issue; a non-Anthropic cross-judge is deferred. (See also the brief disclosure on Page I.) - No claim that any model can stand in for a practicing lawyer. A high score on this battery says something about sustained multi-step reasoning under load; it says nothing about fitness to practice. The task-as-vehicle framing is on Page I.
- OpenAI and Google costs are corrected estimates, not direct-API measurements. The correction uses published per-token rates applied to observed token counts. A direct-API re-run will firm these up.
Where to go deeper
Every number and every exhibit in this issue traces to a source. Here is where to find them.
Colophon
GunnerBench is built and operated by The Ronin Advisory Group LLC. The harness is written in Go against a SQLite database. Every score table, cost figure, and chart in this issue is generated deterministically from the lab's database and the model artifacts it holds. The publication is a projection of a source-of-truth record, not a hand-assembled mock-up.
The model artifacts — the memos, the diagrams — are written by the models themselves to a per-task working directory. The judge reads from the same directory. The database captures the judgment. The exhibit pages present what the database holds. There is no editorial step between what a model produced and what appears in the exhibits; the artifacts are shown as-received.
GunnerBench is built and operated by The Ronin Advisory Group LLC.
The work is the evidence.
© 2026 The Ronin Advisory Group LLC