Limits and non-guarantees
This page lists what the system does not guarantee, what we cannot measure, and four known gaps we deliberately left open. Saltzer and Schroeder's principle of complete mediation requires every access to be checked — the intake point in its default configuration does not satisfy it.
#The five-level vocabulary: how far up we can claim
Receipts has the full table. Here is just the conclusion:
As of September 2026, the furthest we can claim publicly is Decided.
| Level | Status |
|---|---|
| Observed | Met |
| Decided | Met |
| Withheld | Not met by default |
| Mediated | Not met by default (met experimentally under the bwrap wall) |
| Attested | Partly met (four grades: A / B / B′ / C) |
#Complete mediation does not hold
Saltzer and Schroeder's (1975) principle of complete mediation requires every access to be checked. In its default configuration, Vacant's intake point does not satisfy this: if the agent has some other channel for sending the artifact out, the intake point cannot see it.
#Four things it won't do
| Won't guarantee that this AI is reliable | A level belongs to one run, not to one AI. |
| Won't make the AI smarter | Exhibit, 54 cells: 30 stopped at the gate, 0 rescued by retrying. |
| Won't grade work that can't be written as a test | Without acceptance checks that run automatically, there is no referee. |
| Won't make itself impossible to bypass | Only in a run that is locked inside the wall does every call actually pass through the gate. |
#Four things the system cannot catch
#1. Three things chain verification cannot catch
Truncation (no trusted chain head), forks (one event signed into two chains with different contents), omissions (events that were never written down).
A test on 2026-09-19 confirmed the third: a run that never sent a single model request still produced a receipt chain that verified.
#2. Mutual execution has no defense at all against "the suite itself being corrupted"
If the acceptance suite is replaced with one that "passes on load," every vote is honest, every chain verifies, and the system is still delivering garbage. This is a fixed point of the mechanism. R451 → R452 shrank it to one residual (exec smuggling 368/371 → 0/371), but did not eliminate it.
A majority of keys is not a majority of independent parties.
#3. The mathematical upper bound of multi-party execution
Tolerance limit ⌊(k − 1)/2⌋. When a majority colludes, the result flips and honest participants get named, and the mechanism itself does not know which side of the threshold it is on.
At a fixed corruption ratio, the delivery rate does not change by a single digit as k goes from 1 to 7 — for the corruption rate, adding machines buys nothing.
#4. Sybil attacks: we can only write "raises the cost"
Douceur (2002): without a logically centralized identity authority, Sybil attacks are always feasible except under unrealistic assumptions of resource parity. Cheng and Friedman (2005): no symmetric reputation function can resist Sybil attacks — and Vacant's reputation gives every observer the same score, so it is a symmetric function.
Our measurements support this verdict: the anonymous attack is 0.05 × N, linear — at N = 200, total weight 10.0 and target score 0.892. Linear growth has no ceiling.
#Three premises of the measurement layer
#Four known gaps we deliberately kept
#① Identity is still free
The fix branch left 6 structural gaps, and this is the first. vacant_id → stream_id is entirely unverified (registry.py:78-82 recomputes public key ↔ vacant_id, but does not perform a single check on vacant_id ↔ stream_id). Attack PoC, measured: impersonation can inherit reputation, 0.50 → 0.7417.
#② Permanent exclusion: false positives never measured
An identity punished once returned to routing 0 / 368 times in tests. On 2026-08-17 the project explicitly decided to keep this. But for an honest participant misjudged because of the blind spot, the consequence is just as permanent, and that has not been measured.
#③ The refusal primitive can be gamed
Measured: a picky agent's tuned value was 0.253 > the honest participant's 0.204, its reputation +31%, and it was dispatched more work (42.4 → 44.3), but system quality was −4.4% and jobs nobody accepted were +38%.
A count-based score can be gamed by refusing more. Our inference: human intervention gets gamed by the audience, not by the agent.
#④ An agent can delete its own hooks
Measured on 2026-09-20: the hooks in all five agents can intercept shell commands, but an agent can delete its hook configuration. So hooks can only serve as a semantic enhancement, not as a root of trust.
#What we have not measured yet
| Quality of the acceptance tests | How the gate's effect changes when the tests are written well or badly. This is first on our own list of next steps. |
| Omissions | Publish the receipt chain head, so that truncation and omissions become visible too. Second. |
| People | With receipts, do people rely on AI more willingly, and on firmer grounds? Third. |
| A3 attack surface | A3.json exists, but no report analyzes it. |
| Confidence intervals for the B series | Only means; no CIs, no paired tests. |
| Reviewer accuracy 0.7 | That value was chosen by the author, not measured. E12 shows it is the only truly binding constraint in the whole entry-cost table. |
| agentlane | As of 2026-09-22, only code and exhibit evidence; no committed numeric results. |
#Three places known to be bypassable
- Equal-budget comparison — adding machines without changing the rules has zero effect on the corruption rate.
- Human-in-the-loop mediation — gets gamed by the audience effect.
- Trust plug-ins — the agent can unplug them itself.
We write these three down because they decide what to do next: solving them requires replacing the referee or the incentive structure, not adding more signatures.
#Next
- Frequently asked questions: FAQ
- Which statements this project changed: Research history
