Executable acceptance layer

#L3 Separating visible from hidden

Each task's requirement has three parts: the task statement, the public input contract, and the acceptance tests.

The acceptance tests are split into two sets:

A program that passes visible but fails hidden is a false delivery. This is the operational form of the problem "the output looks like the requirement, but isn't the requirement."

#Dynamic audit: did hidden tests leak in?

#Sandbox

The candidate program runs in a separate process:

The test program and the candidate program run in two separate processes, communicating through remote calls that carry a random marker and allow only literal values, so that the candidate program structurally cannot see the test code.

Measured latency (2026-07-09, 24 calls, thinking mode on): median 9.1 seconds, p95 40.0 seconds, max 41.7 seconds, 5.4 minutes in total; Hermes end to end about 78 seconds.

#L4 The acceptance suite is data

#The problem: the gauge can be fed into uselessness

The attack experiment of 2026-09-06 (R451) showed that as long as the acceptance suite is a piece of "trusted code," an attacker can write a stateful variant that defeats the gauge: the gauge scores perfectly in both directions, yet the delivery rate drops −75.20 pp, and the gauge still reports ok.

In a batch of 371 tasks, after the suite was bound into the commit, the plain suite got 371/371 all refused (normal); but the stateful variant made the delivery rate drop 75 percentage points, and the gauge reported ok.

#The fix: SuiteSpec

A suite contains only three things:

1. The dialect
2. The entry-point name
3. A literal list of "input — expected value" pairs

The executor runs only its own deterministic renderer to generate the test code from this data, and the hash of the rendered result can be compared across machines.

After turning the acceptance suite from code into data

#Gauge check: a one-sided guarantee

Before use, a suite must also pass a gauge check: the reference solution must pass, and every known wrong sample must be stopped.

#CONFORM and H-MIX

The two rules share the same gate semantics; they differ in what happens after a failure.

#CONFORM (resample)

Generate candidates in sequence → run visible acceptance on each → let the first one that passes through → at most five → if none pass, refuse delivery. A refused delivery counts as a failure, and the denominator is all tasks.

#H-MIX (revise)

1. The first-round prompt contains the original task text, the list of standard-library modules allowed for import, and one line: "First check your function by hand against the examples in the task."
2. After extracting the code, it first runs a zero-sandbox static precheck (syntax, forbidden imports and attributes, whether the entry point exists); if that fails, it replies with the reason the code couldn't load.
3. Visible acceptance runs in the sandbox; on a pass, it is let through.
4. On a failure it computes a failure signature (failure kind, exception type, message hash). If it matches the previous round, the first time it appends a "try a different approach" prompt; the second time it refuses delivery outright.
5. Per-task cap: 5 calls, 32,000 tokens, 900 seconds.
6. Feedback is presented in the fixed format AssertionError: args=… got=… want=…, truncated if too long.
7. The context keeps only the first user message, the previous model reply and the current feedback.

See the verdict of the five replications

#False delivery rate

The R532 larger-model experiment (836 tasks, 43 blocks, 0 voided):

The gate cut false delivery by roughly 40%, not by half. The thesis originally wrote "halved," which the fact check ruled ERROR E6 — the two rates used different denominators.

#Next

• The gate's rules and verdicts: The acceptance gate CONFORM
• How this layer hooks into your assistant: Product integration layer