2026-09-06 · Records & multi-party execution · Held

After turning the acceptance suite from code into data, the delivery-rate drop went from 75% to 2.7%

An attacker wrote a stateful variant of the gauge: the gauge scored perfectly in both directions, yet the delivery rate dropped 75 percentage points. The fix was to leave the suite with only three things: a dialect, an entry-point name, and a literal list of input — expected value pairs.

SuiteSpecattackgaugesandbox

#The problem: the gauge can be fed into uselessness

2026-09-06, R451.

Stage	Result
Suite bound into the commit, plain suite (counts as passing if it loads)	371 / 371 all refused — normal
Stateful gauge variant	Delivery rate drops 75.20 pp
What the gauge reported	ok (perfect in both directions)

#How the attack works

The original suite was a piece of trusted code. It could:

Read external state.
Change itself or the environment when called.
Behave correctly only on the "known wrong samples."

Multi-party execution does not defend against this — because every vote is honest, and every chain verifies. named-dissent

#The fix: SuiteSpec

Restrict the suite to three things:


1. Dialect	Python / a specific version
2. Entry-point name	`solve`
3. A literal list of "input — expected value" pairs	Pure data

The executor runs only its own deterministic renderer to generate the test code from this data. The hash of the rendered result can be compared across machines.

#Effect

Attack	Before the fix	After the fix
exec smuggling (the candidate program swaps the test code at run time)	368 / 371	0 / 371
generic_gauged (residual)	—	+2.72 pp
weak_oracle (post-hoc upper bound)	—	+4.35 pp
Scope where it is meaningful	—	Only on MBPP+

conformance-gate

#Why the residual +2.72 percentage points can't be removed

This is the definitional boundary of the gauge

What it cannot verify: that these tests cover the requirement. A "weak gauge" (weak oracle) can pass every known check while missing most of the real requirement.

So:

Gauge type	Residual achievable attack	Meaning
generic_gauged	+2.72 pp	The residual actually achievable when there is a gauge
weak_oracle	+4.35 pp	Post-hoc upper bound for a weak gauge

+2.72 percentage points is the width of "the gap in the gauge check." And the check is one-sided — so passing the gauge check is not the same as meeting the requirement.

This is also one of the reasons we saw a 14.4% false delivery rate in R532. gate-cuts-forty-percent

#Regression


Tests	428 passed / 1 skipped (25 new criteria added)
Pinned config digests	Reproduce 15 / 15 counts, bit for bit
E10's review envelopes	180 / 180 still verify

#The general lesson of this case

#Next

What this fix can't solve: named-dissent
The full rules of the gate: conformance-gate
The full spec of multi-party execution: Multi-party execution

The lying key was named all 273 times; honest keys were falsely accused 0 times
k=3, 1 controlled lying key, 5,519 testimonies. Lying cells named 273/273, honest keys falsely accused 0/1,840, contradictory-signature cells ruled correctly 58/58. How strict these numbers are depends on whether you know the mathematical upper bound ⌊(k−1)/2⌋.
The birth of the judging gate: execution instead of opinion, receipts instead of votes
We deleted the AI review committee and replaced it with "run the customer's acceptance tests, deliver the first candidate that passes, refuse delivery if none passes." On three task banks it delivered 4.58 to 19.17 percentage points more, at only 1.5 calls per task.
Detection probability is one product, not three dimensions
We originally claimed that "raising the audit rate cannot rescue cases with a blind spot." Measured: at blind spot 0.5, raising the audit rate from 0.1 to 1.0 cut the attacker's total hits by 82%, p = 5.5e-6. The three factors are terms in the same product.
Six things we mismeasured
Reading the wrong field, network drops counted as reviewer results, an rng seed that degenerates at the limit, treating `defected <= BUDGET` as an equal budget, pretty numbers at an endpoint, and a rule switched off by default — six real measurement traps and their lessons.