Acceptance gate CONFORM

#The rule

The rule is frozen word for word and has no tunable parameters:

1. Generate candidates in order.
2. Run the visible acceptance tests on every candidate in a sandbox.
3. Let the first candidate that passes through, and stop generating.
4. At most five. If none passes, refuse delivery.
5. Refused delivery counts as a failure, and the denominator is all tasks.

#How it replaced the review committee

The gain experiments of 2026-08 showed that a committee-style design (reputation routing, three reviewers, sampled audits, one revision) lost, at the same call budget, to generating the same task five times and taking the majority vote.

The mechanism analysis found the reason:

Reviewers track task difficulty, not whether the answer is right

On 2026-09-03, R440P replaced the whole referee: delete the review committee, and run the customer's acceptance tests instead. In the project documents' words —

Execution instead of opinion, receipts instead of votes.}}

#Measured results

Three task banks, across five replications and four mutually exclusive task sets:

The gate delivers more than a single sample — it holds up across task banks and across replications

The harder the task, the more early stopping is worth

#Equal-budget comparison: the gate beats the vote

This is the cleanest comparison we ran: the same 5 candidates, the same 5 calls; one delivers the first that passes, the other takes a majority vote of the five.

#Why the gate works and reviewers don't

In the two saved batches of five-candidate data, the candidate-pool ceiling was 82.68% (31 tasks with all five wrong) and 85.03% (22 tasks all wrong).

This post gathers the numbers and the refutation record of the whole gate line in one place

#This gate still stands, but it isn't the only path

After the end of 2026-09, the product gained another main line called zero-config (zero-config Vacant). It does only one thing: at the moment the agent says "I'm done," it sends one message back.

Three of the four batches measured no difference. The fourth measured one, and it did not replicate on a different inference engine.

This does not refute the numbers on this page. The reason is that the two are missing different things:

The original document puts it this way: zero-config group C has none of the three parts.

#Sandbox and acceptance suite

Candidate programs run in a separate process: 10 seconds, 128 MiB, a temporary directory as the working directory, and no inherited environment variables. The test program and the candidate program sit in two different processes and communicate through remote calls that carry a random token and allow only literal values, so that the candidate program structurally cannot see the test code.

#The acceptance suite is data, not code

The attack experiment of 2026-09-06 (R451) showed that as long as the acceptance suite is a piece of "trusted code," an attacker can write a stateful variant that breaks the gauge — the gauge scored perfectly in both directions, yet the delivery rate dropped by −75.20 pp, and the gauge still reported ok.

The fix is SuiteSpec: a suite contains only the dialect, the entry-point name, and a literal list of "input—expected value" pairs. The executor only runs test code generated from this data by its own deterministic renderer, and the hash of the rendered output can be compared across machines.

Effect: exec smuggling went from 368/371 → 0/371.

After turning the acceptance suite from code into data

#Preconditions

We write this sentence word for word into every document that reports effectiveness numbers; deleting that block deletes the condition under which those numbers hold.

A related structural finding: in the task banks we used, the hidden tests were always a superset of the visible tests (V ⊆ GT), so "passes hidden but fails visible" is impossible by construction. This empties the sentence "visible filtering is lossless" of content. v-gt-is-not-lossless

#False delivery: the gate really does block some, but not half

R532 larger-model experiment (836 tasks, 43 blocks, 0 voided):

The gate cut false delivery by roughly 40%, not by half. The thesis originally said "halved," which the fact check flagged as ERROR E6 — because the two rates used different denominators.

#Next

• What adding the revision loop does: The executable acceptance layer
• The verdict after five replications: The headline claim lost to its own stability rule
• The token ledger: This is a gain in success rate, not in token efficiency