{"generated":"2026-10-15","lang":"en","entries":[{"t":"What Vacant is","u":"/en/docs/overview.html","g":"Start","s":"Vacant is an accountability layer wrapped around your AI assistant. It doesn't replace your AI. It runs the acceptance tests you wrote, delivers only on a pass, and leaves a signed receipt for every step that you can recompute yourself.","b":"In one sentence The AI says \"done.\" Saying it's done doesn't count. Running it does. Vacant is a layer wrapped around your AI assistant. You hand it a requirement and its acceptance tests; it has your AI do the work, then runs your tests once — on a pass, the work reaches you; on a fail, it is stopped at the gate. Every step of the process is signed; change one cell and the chain breaks at that cell. What it does, and what it doesn't --- --- What it does Decides delivery with the acceptance tests you wrote; signs each verdict into a receipt chain you can recompute yourself; makes participants who did bad work less likely to get the next job. What it doesn't It doesn't make the AI smarter. It doesn't guarantee that this AI is reliable — a level belongs to one run, not to one AI. It doesn't grade work that can't be written as a test. Why \"accountable\" and not \"trust\" The project's definiti"},{"t":"Install","u":"/en/docs/install.html","g":"Start","s":"Vacant is the Python package vacant-network. After installing it, use vacant on to pick, from a menu, an AI assistant already on your computer and launch it under mediation, without touching your persistent settings.","b":"Requirements --- --- Python 3.11 or later Operating system macOS, Linux the sandbox cannot be built on Windows Model channel Any OpenAI-compatible endpoint: LM Studio, Ollama, vLLM, or a cloud API AI assistant A command-line agent Codex, OpenCode, Claude Code, pi… , or an OpenAI-compatible agent you wire up yourself Three ways to connect, with different levels of enforcement This is the single most important thing to know before installing. Vacant's guarantee that \"every call goes through me\" has a scope that depends on how it is connected . Form Enforcement Notes --- --- --- Library / MCP tool Voluntary The agent registers Vacant's MCP server in its own settings and can call tools such as delegate . If the agent doesn't call them, Vacant has no way to step in. Controller Only for child processes it launches itself It delegates, runs acceptance, then launches the downstream agent. The gu"},{"t":"Quickstart","u":"/en/docs/quickstart.html","g":"Start","s":"Walk the whole path with one task that has a known correct answer: write the requirement and its acceptance tests, let the AI assistant do the work, see how the gate rules, learn what each field on the receipt contains, and which three things chain verification cannot catch.","b":"You need to prepare two things The gate runs your tests; it does not ask a model \"does this look right.\" So: 1. A requirement — say clearly what you want. 2. Acceptance tests that can run — what the input is and what the expected output is. What a task looks like The acceptance tests are split in two halves: Who can see it Purpose --- --- --- Visible V The agent can read it The system uses it to decide whether to let the work through or refuse delivery; the revision process also gets only its failure messages Hidden GT Only the after-the-fact scoring can see it Must not enter generation, selection, revision prompts or stopping conditions A program that passes visible but fails hidden is a false delivery. This is the operational form of the problem \"the output looks like the requirement but isn't the requirement.\" Here is a pitfall we hit ourselves that is worth knowing up front: in the t"},{"t":"Accountability layer","u":"/en/docs/accountability-layer.html","g":"Core concepts","s":"In this project, \"accountable\" is not the same as \"trust.\" This is Vacant's core definition. It was changed five times during the research, and each time the claim got smaller — this table is the only version that is valid now.","b":"The current definition Intake point: a delivery without a verifiable receipt is not accepted. The enforcement point is at acceptance time , not at run time. When the downstream side receives a delivery, it recomputes the hashes on the receipt and verifies the signature, and a receipt can be consumed only once . In this project's precise terms, it is \" a record that can be re-verified afterwards, in which tampering can be detected .\" Why we don't say \"trust\" The classic definition of trust includes \"no need to monitor.\" And monitoring is exactly what Vacant does — it is that monitoring layer. Saying \"it increases trust\" would be read as \"you can check a little less,\" which is the opposite direction. An independent review in 2026-07 settled this. Its conclusion was one sentence: What this architecture currently increases is accountability, not trust. It also listed three conditions that mu"},{"t":"Receipts","u":"/en/docs/receipt.html","g":"Core concepts","s":"A receipt is a \"tamper-evident\" signed record, but it comes in five levels, and every receipt must declare for itself which level it reaches. An audit on 2026-09-20 found that \"refused delivery\" was only a flag — since then we have stopped saying \"stopped at the exit\" at all.","b":"The five-level vocabulary On 2026-09-20, an internal audit found that in one batch of experiments, \"refused delivery\" was only a flag and an exit code — the outer script did not read the exit code, and copied the rejected files out anyway. Of 391 audited cells, 7 of the 59 refused-delivery cells had their files copied out and scored. After that audit, the project switched to a five-level vocabulary to describe \"what the system has proven,\" and requires each level to declare its own coverage. Level What the system has proven Status as of 2026-09 --- --- --- Observed It saw certain behavior, traffic or artifacts Holds : the conversation seen by mediation has no gaps all 2,755 rounds match up, 195/195 chains verify Decided It independently judged whether this delivery passes or not Holds : when the agent claims it is done, the system can still rule refused delivery and sign it Withheld An a"},{"t":"Acceptance gate CONFORM","u":"/en/docs/conformance-gate.html","g":"Core concepts","s":"A rule frozen word for word: generate candidates in order, run the visible acceptance tests on each, let the first one that passes through; at most five, and if none passes, refuse delivery. Refused delivery counts as a failure and the denominator is all tasks — this is the most important half of the rule.","b":"The rule The rule is frozen word for word and has no tunable parameters: 1. Generate candidates in order. 2. Run the visible acceptance tests on every candidate in a sandbox. 3. Let the first candidate that passes through, and stop generating. 4. At most five. If none passes, refuse delivery . 5. Refused delivery counts as a failure, and the denominator is all tasks. How it replaced the review committee The gain experiments of 2026-08 showed that a committee-style design reputation routing, three reviewers, sampled audits, one revision lost, at the same call budget, to generating the same task five times and taking the majority vote. The mechanism analysis found the reason: Metric Value --- --- Reviewer vote accuracy 0.7552 The zero-intelligence baseline \"rule everything as pass\" 0.7522 Gap +0.29 percentage points Share of single-round revisions that \"had no chance to fix anything\" 93 / "},{"t":"Attribution","u":"/en/docs/attribution.html","g":"Core concepts","s":"For \"who did what\" to hold, three things must be true at once: key custody, deterministic sampling, and memory that follows the event stream. If any one is missing, attribution stops at \"some key signed this entry\" instead of \"some participant did this.\"","b":"The three conditions for attribution Condition Current state Evidence --- --- --- Key custody Does not hold In the current product path the private key is a plaintext file readable by the same OS user, with no hardware protection Deterministic sampling Partly holds Deterministic replay is a real anchor, but the sampling seed is hard-coded and publicly predictable Memory follows the event stream Holds The accounting unit of reputation is the event stream; if you switch the model or wipe the memory, the credit does not carry over This sentence appears again in , where its concrete consequences are laid out. Twin identity vacant_id is derived from the public key, not looked up: So anyone can recompute the identifier from the public key, without querying a central registry . This is the concrete form of the principle \"no central referee.\" Three breaks in the binding The independent review 20"},{"t":"Zero-config Vacant","u":"/en/docs/zero-config.html","g":"Core concepts","s":"Works as soon as it's installed, with no configuration at all. It does only one thing: within the same session, it sends one message back. The price is that it lacks three parts — an executable signal, the power to retry, the power to refuse delivery. Three of the four batches measured no difference.","b":"Zero-config is the v0.9.0 product line . Its design goal is not to do a lot, but to let you work as usual once it's installed — no contracts to write, no settings to add, no change of workflow. {{note:warn First, what this page is about and what it isn't This page is titled \"zero-config,\" which makes it easy to assume it replaced the . It didn't. The difference between the two is not old version versus new version; it is the mechanism itself . CONFORM owns generation: every candidate runs the visible acceptance tests, a failing one is resampled, at most five, and if none passes, delivery is refused. Zero-config does not own generation; it only sends one message back, within the same session, at the moment the agent says \"I'm done.\" This page holds zero-config's own experimental results. Of the four batches, three measured no difference. The fourth measured one, but it did not replicate o"},{"t":"Accountable trace","u":"/en/docs/accountable-trace.html","g":"Core concepts","s":"The general intake point answers \"does this deliverable pass or not,\" but it cannot say where it fails, which step got it wrong, or who got it wrong. The trace cuts down to the step: every conclusion has two layers, and only the layer obtained by a re-run flip can carry reputation consequences.","b":"The thesis's one-sentence positioning is \" know who did what , and trace it back when something goes wrong.\" and the general intake point cover only the second half, \"when something goes wrong,\" and they lack three things: It cannot say where it fails ClaimResult has no location , which step got it wrong there is no step record , or who got it wrong there is no actor , and the feedback is a piece of generic text. On 2026-09-24, after reviewing the version built entirely on the intake point, the human said, in their own words: \"What this version of yours does is give the local machine several validators that can go and verify things; it is not something that assigns blame and catches the point of error .\" original: 「你現在的這個版本是讓本機多個可以去驗證的驗證器， 並非是咎責且抓出錯誤點 的東西。」 The trace fills in those three things. The same round settled four points: the subject of blame should be both the step and the acto"},{"t":"Architecture","u":"/en/docs/architecture.html","g":"Architecture","s":"Vacant has seven layers: L0 identity, L1 signed record, L2 accountability layer, L3–L4 executable acceptance, L5 observation, L6 product integration. Four mechanisms answer four kinds of cheating, and each one has a failure condition we measured.","b":"Positioning The project's core document puts what Vacant does in one sentence: It does not make AI trustworthy. It turns \"who did what, who checked it, and how carefully they checked\" into a record that cannot be changed afterwards , and it makes participants who did bad work get less work. After September 2026, this positioning became more concrete as an intake point : a delivery without a verifiable receipt is not accepted. This positioning deliberately avoids two common misreadings: - Vacant is not another agent framework. It does not decide for the agent how to reason or which tools to call. - Vacant is not an enforcement layer wrapped around the agent that is guaranteed to intercept everything. Saltzer and Schroeder's 1975 principle of complete mediation requires every access to be checked, and the intake point in its default configuration does not satisfy that. Seven layers Layer N"},{"t":"Identity and signed record","u":"/en/docs/identity-logbook.html","g":"Architecture","s":"Every resident holds an Ed25519 key pair, and vacant_id is derived from the public key, so no registry needs to be looked up. Each event-chain entry has eight fields, the chain is append-only, and every signature covers the bytes of canonical JSON.","b":"L0 Cryptographic identity Every participant the project calls them \"residents\" holds an Ed25519 key pair. vacant_id is derived Anyone can recompute the identifier from the public key, without querying a central registry . This is the concrete form of the design principle \"no central referee\" — no single table is authoritative. Disk layout: trust store and capability store are separate Store Contents Owner --- --- --- Trust store Private key, public key, event chain, reputation snapshot, capability card Gateway Capability store The agent's configuration and memory Agent The design principle is \" identity is not exposed to the brain \" — the agent's reasoning process cannot touch the signing key. Capability card halo Every resident carries its own capability card as an announcement. The Registry is only an aggregate query interface and reputation index over these announcements; it is not a "},{"t":"Registry and reputation","u":"/en/docs/registry-reputation.html","g":"Architecture","s":"Reputation is recorded on a four-tuple (event stream, branch, model substrate, task family), using a five-dimensional Beta distribution plus UCB routing. Newcomers have a probation cap. The side effect of a penalty is permanent exclusion — of 368 seeds, 0 came back.","b":"The accounting unit of reputation Reputation is not recorded on \"an agent.\" It is recorded on a four-tuple : Component Why it is in the key --- --- stream event stream Credit follows memory , not body . If the same key wipes its memory and starts a new genesis event, it gets a new stream, and its credit naturally resets to zero. branch branch Parallel branches of the same event are counted separately substrate model substrate Stops \"credit built up on model A being used to cover for deliveries from model B\" family task family Separates \"this participant is reliable overall\" from \"this participant is good at this kind of task\" Task family was added on 2026-08-07, corresponding to the \"who knows what\" distinction in research on transactive memory systems. Five-dimensional Beta Every reputation cell has one Beta α, β distribution in each of five dimensions, with prior Beta 1, 1 : Dimension "},{"t":"Peer review and audit","u":"/en/docs/peer-review-audit.html","g":"Architecture","s":"Every delivery is reviewed by three peers, with no whitelist and weights that arise endogenously from reputation. An audit is not asking a model again but a deterministic rerun in a sandbox — because the blind spot is not zero, and adding more reviewers adds no independent information.","b":"Peer review K = 3 Reviewers are chosen by sorting on a hash of the task hash and the resident's name , and no one reviews themselves . Before results are written into the registry they must pass four checks see . No whitelist Anyone who has announced themselves can review. The weight is decided endogenously by the reviewer's own reputation: score_r and obs_r are the reviewer's reputation and observation count on the same model substrate . A newcomer's vote is very light, but they do have a vote . The cost of having no whitelist: vote stuffing Three tiers of down-weighting respond to it: Tier Condition Cap --- --- --- ① Self-reported same controller The k-th reviewer under the same controller reviewing the same target 0.1 / k ② Same origin inferred from behavior Two reviewers agree at a rate ≥ 0.9 on \"discriminating tasks,\" having co-reviewed ≥ 5 such tasks 0.1 / k ③ Unproven reviewer Own"},{"t":"Executable acceptance layer","u":"/en/docs/executable-acceptance.html","g":"Architecture","s":"The visible tests V decide release; the hidden tests GT are used only for scoring afterwards. The sandbox makes the test code structurally invisible to the candidate program. The acceptance suite is data, not code — otherwise an attacker can make the gauge score perfectly in both directions while delivering garbage.","b":"L3 Separating visible from hidden Each task's requirement has three parts: the task statement , the public input contract , and the acceptance tests . The acceptance tests are split into two sets: Who can see it Used for --- --- --- Visible V The agent can read it The system uses it to decide release or refused delivery; the revision flow also only gets its failure messages Hidden GT Only scoring afterwards Must not enter generation, selection, revision prompts or stopping conditions A program that passes visible but fails hidden is a false delivery . This is the operational form of the problem \"the output looks like the requirement, but isn't the requirement.\" Dynamic audit: did hidden tests leak in? Version Approach Result --- --- --- v1 Keyword matching 90 false positives v2 Criterion changed to \"who wrote this text\" R460R and R529, all 67 blocks with zero violations ; an independent "},{"t":"Multi-party execution","u":"/en/docs/peer-execution.html","g":"Architecture","s":"What is decentralized is execution and testimony, not judgment. The reason: disagreement in opinion carries no information; disagreement in execution does. This path tolerates at most ⌊(k−1)/2⌋ corrupted executors; past a majority, the result flips.","b":"Why \"run each other, don't review each other\" An early design that was abandoned was \"decentralized mutual review.\" Multi-party execution rewrote it as running each other's work instead of reviewing it: what is decentralized is execution and testimony, not judgment. The reason fits in one sentence: Disagreement in opinion carries no information; disagreement in execution does. Acceptance is a deterministic program, so honest executors necessarily get the same result. If a disagreement appears, the only possibilities are that someone didn't really run it, or that someone signed content different from their own execution result — both are attributable faults . Three things 1. Execute and attest : each executor runs acceptance with its own renderer and sandbox, and signs into its own event chain the hashes of the task, the candidate, the suite and the rendered result, along with the visible"},{"t":"Product integration layer","u":"/en/docs/product-integration.html","g":"Architecture","s":"Three integration shapes, three strengths of enforcement. `vacant run` treats \"the process ends\" as the delivery point; the sandbox wall gives bwrap separate namespaces; the exit separates release/publish capabilities. An agent can delete its own hooks, so hooks reinforce — they are not a root of trust.","b":"Three integration shapes Shape Enforcement Description --- --- --- Library / MCP tool Voluntary The agent registers Vacant's MCP server in its own configuration and can call tools such as delegate . If the agent doesn't call them, Vacant has no way to step in. Controller Only for child processes it launches itself Delegate first, check acceptance, then start the downstream agent. The guarantee only covers processes launched through this controller; the same OS user can bypass it and run the agent directly. Harness that owns the loop Fully enforced Generation, acceptance and shipping are all controlled by it. The gain experiments' runner is of this kind. The controller's source-code comment states this boundary: the guarantee only covers child processes launched through this controller. vacant run : the end of the process as the delivery point Added on 2026-09-18. It rests on one observat"},{"t":"Native platform and silent no-ops","u":"/en/docs/native-platform.html","g":"Architecture","s":"Putting the decision point inside the agent process does not become \"a smaller effect\" when the platform doesn't cooperate; it becomes a silent no-op — a complete case record, a signature chain that verifies, zero verdicts. Three gaps, and one decision that moves the loop into the parent process.","b":"As of 2026-09-29 the status of this line is a design decision, effect not measured . This page records the three gaps it did measure. Three gaps W1 — channel mediation only recognizes two kinds of key variable SECRET_VARS in envmap.py dropped OPENROUTER_API_KEY , and discover_keys reads only two: openai and anthropic. The measured consequence: opencode's openrouter provider exited with rc=1 locally within 1.2 seconds, requests_seen = 0 , without sending anything over the wire even once . After switching to an openai-compatible provider on a fixed port, requests_seen became 2 — channel mediation works for opencode, but opencode still exits with a nonzero code. This path was not completed. W2 — opencode has no stop-equivalent event The opencode plugin generated by vacant install registers only chat.message , tool.execute.before and tool.execute.after ; pi's template of the same name has as"},{"t":"Experimental methods and verdict values","u":"/en/docs/experiments.html","g":"Experiments","s":"Four families of experimental method, the six states a verdict can take, the four rules declared in advance, and the four analysis disciplines that were written only after re-verification. This page is the reading precondition for every number on the site.","b":"Two axes From 2026-08-17 on, the project formally split all of its experiments into two axes. This was the most important structural change in the whole study — before it, every experiment we had accumulated was asking \"was the bad stuff blocked?\", and not one was asking \"with this layer added, does what gets delivered fit the requirement better?\" Axis Question Method --- --- --- Defense axis When there are participants who deliberately deliver wrong work, how much can these rules block? Mainly mechanism simulation Gain axis When requirements can already be compiled into executable acceptance tests, how much does the delivery success rate rise, and at what token cost? Paired experiments on real models Four method families 1. Mechanism simulation E1–E9, E12–E24, A1–B5, M1–M5, S1–S3 What it answers : under these rules, what a given attack strategy or mechanism parameter is worth. What is r"},{"t":"Defense axis","u":"/en/docs/defense.html","g":"Experiments","s":"How much work an attacker can get. Of this axis's 12 claims in the project index, 3 were refuted by ourselves and 3 were overstated — but the mechanism that holds off bad actors does work, and what works is not \"catching them\" but \"starving them.\"","b":"What this axis answers In an agent population that contains participants who deliberately deliver wrong work, how far can signed records, reputation routing, peer checks, sampled audits, and penalties reduce the chance that a wrong delivery is accepted? The most effective one: being seen, not being caught This is the defense axis's strongest positive result, and it is not \"catching them.\" Group Work received over 9,600 rounds --- --- 215 seeds that were never caught 7 – 24 jobs Control group that never violates 1,290 jobs In other words: bad actors who were never caught received only 0.57% – 1.84% of an honest participant's workload. The conclusion in one sentence: what stops attackers is \"being seen,\" not \"being caught.\" Reputation routing alone is enough to starve them, without catching them even once. Source: runs/s5_v1_FINDINGS.md Entry cost: neither path should be taken 2026-07-26, "},{"t":"Gain axis","u":"/en/docs/gain.html","g":"Experiments","s":"This is the axis we spent the most effort on, and the one where we refuted ourselves the most. The headline was \"AI reviewers lose to majority vote\"; the conclusion is \"executable acceptance gate + resampling works; the revision loop's extra gain is Same sign, unresolved.\"","b":"What this axis answers Given that requirements can already be compiled into executable acceptance tests: how much gain do the acceptance gate, resampling, and failure-feedback revision each bring to the delivery success rate, and at what token cost. The start: the committee design lost The early G series exploratory Equal-budget comparison same tasks, same number of calls : Comparison n Result p --- --- --- --- ON vs OFF5 371 tasks 347 82.42% vs 83.57% 0.5572 OFF5 vs OFF not equal budget 349 83.09% vs 78.51% 0.0090 Key finding : the OFF5 arm has accepted = True hard-coded — it has no path to refuse . So the comparison above is really \"an arm with a free gate\" against \"an arm that is forbidden to refuse.\" R437, the decisive three-arm run Three criteria set in advance frozen 2026-08-24/28 : Criterion Verdict --- --- ① OFF failure rate falls within 20–60% ✓ 26.44% Wilson 20.4, 33.4 — the ga"},{"t":"Records & multi-party execution","u":"/en/docs/records.html","g":"Experiments","s":"194 receipt chains and all 9,841 records verify; the lying key was named 273/273, and honest participants were falsely accused 0/1,840. But chain verification cannot catch truncation, forks, or omissions, and mutual execution has no defense at all against the acceptance suite itself being corrupted.","b":"Receipt chain verification Batch Chains Records Result --- --- --- --- R460R five replications — 6,674 All passed R529 four sets across task banks — 3,167 All passed Total 194 9,841 All passed independent signature verification R532 larger model 86 3,895 All passed ABPI audit 195 — All passed R454 multi-party execution — 5,519 testimonies 0 signature-verification failures E10 after the channel-separation revision — 180/180 reviewer envelopes Still verifiable Three things are verified: sequence numbers are contiguous, each record's previous hash matches, and the Ed25519 signature is valid. Coverage is only CONFORM and the three H arms — the OFF and OFF5 arms have no receipts . V/GT leakage audit Version Scope Result --- --- --- v1 — 90 false positives v2 R460R 30 blocks + R529 37 blocks Zero violations v2 independent procedure 2,497 submission texts, 164,963 comparison items Zero hits wit"},{"t":"Evidence ledger","u":"/en/docs/evidence-ledger.html","g":"Evidence & reproduction","s":"58 public claims, each with a verdict, supporting figures and a way to recompute them. The refuted ones, the overstated ones, the ones where no difference was measured — all stay on the table. A system that claims to be accountable has to be accountable to itself first.","b":"Where this table comes from Each claim carries four things: 1. Original wording — what was said publicly at the time. 2. Supporting figures — the numbers behind it, with denominators. 3. Verdict value — one of six states. 4. Corrected — if the verdict is \"Refuted\" or \"Overstated\", what the correct statement is. Counts The full ledger Click any row to expand its supporting figures, source-record path and how to recompute it. Where the verdicts cluster This section covers only the 34 claims in the project's official index every count in parentheses below uses these 34 as the denominator . The 24 repo verdicts are in the full ledger above; use the filter bar to slice by \"axis\" or \"verdict\" to see them. Refuted 6 of 34 — all on the defense axis and \"equal budget\" ID Claim Why it was refuted --- --- --- pulse.crossover Once the blind spot exceeds 0.5–0.75, pulse does worse than sustained misb"},{"t":"Reproduction","u":"/en/docs/reproduce.html","g":"Evidence & reproduction","s":"The recipes, one by one: mechanism simulation, the real-model suite, zero-API offline replay, and receipt-chain verification. Most statistical conclusions can be rerun with zero-API recomputation scripts, without calling any model.","b":"Why most results can be rerun at zero cost Every source record of the gain experiments is written to disk per task in rows.jsonl , with five fields: arm / task_id / delivered / hidden_pass / calls_used . This means the statistical conclusions don't need the model to be called again : Mechanism simulation Fully deterministic: the rng seed = f'{seed}:{config_digest}' , so the same parameters always give the same results . Real-model suite --- --- Task bank EvalPlus MBPP+ v0.2.0 378 tasks, sha256 pinned Quality check compile_check t.check answer , outside the arm Pairing key task_id = sha256 prompt + tests , stable across arms Statistics McNemar exact test + fixed-seed bootstrap 95% CI + statistical power Resume Keyed by arm, i ; an interruption doesn't mean starting over Receipt chain and offline viewer We also host the offline single-file viewers on the site, so you can download them and "},{"t":"Research history","u":"/en/docs/journey.html","g":"Evidence & reproduction","s":"From \"a responsibility structure that agents adopt voluntarily\" to \"a delivery without a verifiable receipt is not accepted.\" In nine months we changed the definition five times, refuted six of our own claims, and gave up three attack names — this table is that path.","b":"Why the history belongs in the docs Because a system that claims to be accountable must first be able to be accountable to itself. This is design principle six, and it is the only thing this project can prove about itself. If we put only the final version of the conclusions here, outsiders could not tell whether our statements were \"thought through\" or \"just never questioned.\" So every refuted path stays. Milestones Date 2026 Event --- --- April Project proposal: \"An inter-agent collaboration protocol based on a responsibility structure.\" The core metaphor: today's agents are like \" doctors with no license, no record, and no consequences \" — however capable, nobody dares let them operate. 04-28 First advisor meeting. The topic was Harness Engineering glossed in Chinese as \"pommel-horse mechanism,\" original: 「鞍馬機制」 . \"Why does nobody use agents?\" → the pain point is credibility. The advis"},{"t":"Limits and non-guarantees","u":"/en/docs/limits.html","g":"Limits","s":"This page lists what the system does not guarantee, what we cannot measure, and four known gaps we deliberately left open. Saltzer and Schroeder's principle of complete mediation requires every access to be checked — the intake point in its default configuration does not satisfy it.","b":"The five-level vocabulary: how far up we can claim has the full table. Here is just the conclusion: As of September 2026, the furthest we can claim publicly is Decided. Level Status --- --- Observed Met Decided Met Withheld Not met by default Mediated Not met by default met experimentally under the bwrap wall Attested Partly met four grades: A / B / B′ / C Complete mediation does not hold Saltzer and Schroeder's 1975 principle of complete mediation requires every access to be checked . In its default configuration, Vacant's intake point does not satisfy this: if the agent has some other channel for sending the artifact out, the intake point cannot see it. Four things it won't do --- --- Won't guarantee that this AI is reliable A level belongs to one run, not to one AI. Won't make the AI smarter Exhibit, 54 cells: 30 stopped at the gate, 0 rescued by retrying . Won't grade work that can't"},{"t":"FAQ","u":"/en/docs/faq.html","g":"Limits","s":"How it compares with other things, whether you can use it on your own project, how it differs from LangSmith / OpenTelemetry / blockchains, and the questions we have been asked many times that we can only answer with \"we don't know.\"","b":"What it is How is accountable different from trustworthy? Trustworthy is a judgment about a property of the system; accountable is a judgment about a property of the record. Our precise definition is: the traceability and verifiability of events signed by a specific key , with no psychological sense of trust included. Why not \"trust\": the classic definition of trust includes \"no need to monitor,\" and monitoring is exactly what Vacant does . Saying \"it increases trust\" gets read as \"you can check a little less,\" which is the opposite direction. An independent review in 2026-07 settled this, and its conclusion was one sentence: What this architecture currently increases is accountability, not trust. has the full explanation. Why is it called Vacant? In English it means \"empty, unoccupied.\" We use it to name the gap in the agent ecosystem that has not yet been filled: a responsibility layer"},{"t":"Corrections log","u":"/en/docs/changelog.html","g":"Limits","s":"Every claim that was downgraded or refuted, every number that was corrected, and every sentence we were asked to delete, all in chronological order. From \"blocked at the exit\" to KS-1, this is the part of the whole research most worth reading.","b":"Why this page exists Because a system that claims to be accountable must first be able to be accountable to itself. This page is the form in which the project puts that into practice: every downgraded claim, every corrected number, every deleted phrasing, in chronological order. The full cross-reference is in the . 2026-10-07 — Site data audit We checked the site's numbers one by one against their sources the project's _index/claims.json , the thesis, and the Vacant repo's decision and conclusion documents , and changed the following: - The denominators for zero.reminder_default_off were attached to the wrong batch : the denominators written next to \"share of correct answer files about 82% → 65%\" A 119, C37 161, C37R 211 belong to the 10-02 Colab v3.7 batch, not the 09-26 local batch. The two batches are now written separately: the local batch uses the decision document's original wordin"},{"t":"Six verdict values: how to say \"not measured\" without saying \"no difference\"","u":"/en/blog/verdict-values.html","g":"Method","s":"Held, Refuted, Overstated, No discernible difference, Same sign, unresolved, Not re-verified — each of these six terms has a strict definition. We keep them apart so that \"I didn't detect it\" is never read as \"I tested it, and there's nothing.\"","b":"This is the most important set of terms on the whole site. Every public claim must land in one of these cells , and there is no seventh. Why split them so finely Because \"not measured\" and \"measured, and nothing there\" are two different things, and reports often write them as the same sentence. - Measured, and nothing there → you can say \"this has no effect under these conditions.\" That is an informative conclusion. - Not measured → you can only say \"I can't tell them apart at this scale.\" That is not a conclusion; it is the limit of the data. Our R529 happens to contain both situations at once, and the difference is plain to see: Figures Verdict --- --- --- H-MIX − OFF +7.96 pp , Holm p = 2.02 × 10⁻⁸ Held H-MIX − CONFORM +1.12 pp , Holm p = .3409 Same sign, unresolved The same experiment, the same data: one Held, one unresolved. The only difference is whether the size of the difference "},{"t":"Preregistration: why we write down the criteria before running the experiment","u":"/en/blog/preregistration.html","g":"Method","s":"Criteria, statistical methods and the four state rules are all frozen and timestamped before the run starts. This post explains how we preregister, and why \"adding a criterion after the fact\" doesn't count in this research.","b":"Why this is needed Because choosing criteria after the fact is one of the most common forms of statistical cheating , and it doesn't require anyone to lie. All you have to do is, after seeing the results, pick a threshold that \"was going to pass anyway\" and write it into the report — nobody in the whole process does anything wrong, but the conclusion is false. We ran into this once. §7 of R440P has a premise that reads \"visible filtering is lossless.\" It is a criterion , written in the decision document, and it looks fine. Nine days later R440Q refuted it: In LCB v1, 91/91 hidden test sets are supersets of the visible tests V ⊆ GT . So \"would pass hidden but fails visible\" cannot happen , by construction. This isn't \"no counterexample was measured\"; it is \"this proposition is always true on this task bank.\" A criterion that is always true has zero statistical power. The four state rules "},{"t":"We sent AI to refute our own conclusions, and they succeeded","u":"/en/blog/adversarial-recheck.html","g":"Method","s":"In early 2026-08, we handed six headline conclusions about the pulse attack to independent agents, with one instruction only: \"prove it wrong.\" The result: three refuted, two downgraded, and only two held up.","b":"Results Original conclusion Verdict --- --- Temporal structure barely affects total hits 1.81× spread across 15 cells Overstated The blind spot matters far more than the attacker's parameter choice Overstated Beyond a blind spot of 0.5–0.75, pulsing does worse than sustained misbehavior Refuted Raising the audit rate cannot rescue cases with a blind spot Refuted Caught once = removed permanently Overstated At equal budget, crude strategies beat pulsing Refuted Of the six, three were refuted and two downgraded. The two positive conclusions that were kept have both been written into our analysis disciplines as rules. Why it is so easy to get wrong Because when we wrote the report, we subconsciously treated two related factors as orthogonal dimensions . Three sets of examples: What we thought What it actually is --- --- Temporal structure vs total hits Total hits = exposure × efficiency , a"},{"t":"Six things we mismeasured","u":"/en/blog/measurement-traps.html","g":"Method","s":"Reading the wrong field, network drops counted as reviewer results, an rng seed that degenerates at the limit, treating `defected <= BUDGET` as an equal budget, pretty numbers at an endpoint, and a rule switched off by default — six real measurement traps and their lessons.","b":"This post gathers in one place everything in the project that we mismeasured . Every one of them really happened, and every one of them nearly became a false positive or negative conclusion. 1. Reading the wrong field produced a false result The first analysis of the E10 real-model experiment : Reading audit.passed Using the outside-the-arm check --- --- --- off arm 0 / 11 7 / 12 on arm 5 / 10 5 / 10 Difference Significant on far ahead Disappears The right way : the quality check must be independent of the arm — judge independently with compile_check t.check answer , outside the arm . This one is kept in a comment in examples/realmodel_suite.py as a classic trap.}} 2. Network drops counted as \"reviewer pass\" 2026-06-30, re-verification of crowd routing. The first round of a2hard printed ✓ , but a per-call I/O audit caught this: - Calls 20–23 exactly good_1's 4 peer reviewers ran into Con"},{"t":"An attacker that only slanders and never breaks a rule got 95.4% of the work","u":"/en/blog/collusion-scores-zero.html","g":"Defense axis","s":"All three layers of anti-vote-stuffing defense were in place, yet a collusion group that never delivered anything bad and only reviewed others took almost all of the routing — and its score is 0. The harm was not underestimated; it was measured as zero.","b":"On 2026-08-07, we ran a set of \"new attack surface\" experiments A1–A3, B1–B5 . The result of one of them is the claim in the whole project that most needs to be cited with care . Design --- --- Scale 6 reviewers, 6 bug types expert accuracy 0.90 vs non-expert 0.35 Routing UCB over agent, family cells family declaration Declared by the delivering side in note_head ; reviewers can only transcribe it A series 12 seeds; B series 10 seeds Statistics Means only, no CIs, no paired tests Three layers of anti-vote-stuffing defense We have no reviewer whitelist because a whitelist needs someone to issue it, and that person becomes someone who must be trusted . The price is vote stuffing, so we set up three layers of down-weighting: Layer Trigger Weight cap --- --- --- ① Self-reported same-origin Self-reports the same controller; the k-th reviewer on the same target 0.1 / k ② Behavior-inferred same"},{"t":"Why \"permanent exclusion\" is really the product of two reasonable patches","u":"/en/blog/slash-halves-both.html","g":"Defense axis","s":"Measured: of 368 penalized seeds, 0 returned to routing. We said \"caught once = removed permanently\" — but the real mechanism is not a clock; it is that slash both halves the mean and doubles the observation count, and the observation count is the exponential coefficient of the return time.","b":"Here is the experimental result: across 368 simulation seeds that were actually penalized, 0 returned to routing . Our conclusion was written as: Caught once = removed permanently. The phenomenon in that sentence is right, but the mechanistic explanation is wrong. And a wrong mechanistic explanation leads to a wrong fix, so it is worth tracking down to the end. First, how extreme it is Parameters Share returning to routing --- --- Current f = 0.5, λ = 1 0 / 368 Move only the mean, not the observation count λ = 0 214 / 368 58.2% f = 0.7, λ = 0 308 / 368 f = 0.7, λ = 1 0 / 368 f = 0.9, λ = 0 309 / 368 f = 0.9, λ = 1 308 / 368 The closed form: not \"a long wait,\" but \"unreachable\" The penalty can be written in closed form as: Taking m = 0.9, target = 0.95, c = 0.3, the total observations each setting needs log₁₀ are: n before the penalty f = 0.5, λ = 0 f = 0.5, λ = 1 current f = 0.9, λ = 0 f"},{"t":"How \"temporal structure doesn't matter\" went wrong: breaking the aggregate apart","u":"/en/blog/aggregate-decomposition.html","g":"Defense axis","s":"We measured total hits across 15 cells of attack strategies that differed by only 1.81×, then wrote \"temporal structure barely affects the result.\" Broken apart: exposure spans 6.06× and efficiency spans 8.45× — attack capability differs by 8.45×.","b":"What we originally measured The attack strategy is the \"pulse attack\": discharge B consecutive bad deliveries, charge R clean deliveries, repeat. We swept 15 burst, recover combinations, at blind spot 0.5: Value Cell --- --- --- Fewest hits 2.8333 1, 20 Most hits 5.1333 8, 20 Span 1.81× Based on this number, we wrote in the report: The temporal structure of the attack barely affects the total number of hits. That sentence is wrong. Or, more precisely, it is a reasoning error , not a calculation error. What the re-verification agent found It split \"total hits\" into two quantities: Span --- --- Exposure times routed 6.06× Efficiency hit rate per routing 8.45× The two multiplied 1.81× The correct statement At blind spot 0.5 we could not measure an effect of temporal structure on total hits, but that is absence of evidence . Once hits are split into \"times routed × hit rate per routing,\" the"},{"t":"Detection probability is one product, not three dimensions","u":"/en/blog/blindspot-is-a-product.html","g":"Defense axis","s":"We originally claimed that \"raising the audit rate cannot rescue cases with a blind spot.\" Measured: at blind spot 0.5, raising the audit rate from 0.1 to 1.0 cut the attacker's total hits by 82%, p = 5.5e-6. The three factors are terms in the same product.","b":"What we originally claimed Our intuition went like this: If audits and reviews share a blind spot, then no matter how high you push the audit rate, you cannot make up for the loss the blind spot causes — because what is missed is always missed. The numbers it rested on: Attacker hits Full-block rate --- --- --- Blind spot 0, audit rate 1.0 0.0 100% Blind spot 0.5, audit rate 1.0 0.9667 56.67% So we wrote: \"Raising the audit rate rescues the zero-blind-spot case, but not cases with a blind spot.\" What the re-verification agent found It swept the full grid and then wrote: Detection probability is a single product 1−blind spot ×sampling rate×accuracy , and the three share one axis. Sampling rate and blind spot can substitute for each other. The blind spot changes the defense's ceiling residual β/ 1−β , not whether the sampling rate is useful. Measured numbers Condition Result --- --- Blind "},{"t":"The \"equal budget\" trap: our ranking flipped completely","u":"/en/blog/equal-budget-must-bind.html","g":"Defense axis","s":"The criterion was written as defected <= BUDGET, an upper bound that always holds, so it could not catch \"this arm was never bound at all.\" As a result, whitewash misbehaved 2.5 times as much and we read it as \"the strategy is cheaper.\" After switching to == and verifying, the ranking flipped.","b":"What we wanted to do E19 set out to answer a very intuitive question: Given the same misbehavior budget, which timing pattern is the most economical for an attacker ? So we set a cap: the total number of acts of misbehavior cannot exceed 12. Then we compared the \"efficiency\" of five attack strategies. What we measured Arm Efficiency under the criterion Bound? --- --- --- whitewash misbehaves all the time, switches identity when caught 8.1667 29 / 30 sybil 6.1333 30 / 30 pulse 5,20 4.30 1 / 30 pulse 3,10 4.2333 1 / 30 patient misbehaves slowly 3.7333 1 / 30 Our conclusion: Under an equal budget 12 each + blind spot 0.5, whitewash at 8.17 actually beats pulsing at 4.23 . \"So a crude continuous attack is more economical than a carefully paced pulse.\" That sentence reads as fairly insightful. Where it went wrong So whom did that budget of 12 actually bind? Arm Share of seeds that reached the"},{"t":"Pulse attack: not new, and even our headline number was already old news","u":"/en/blog/pulse-attack.html","g":"Defense axis","s":"Discharge B, charge R, repeat. This model had already been done in 2005 (Srivatsa et al.'s strategic oscillation); on 2026-08-06 we withdrew the \"novel attack\" claim.","b":"The model The attack strategy is a \"pulse\": Parameterized as B, R . Three existing strategies are special cases of it: Existing strategy B, R --- --- patient sustained, slow 1, ∞ whitewash misbehaves all the time ∞, 0 Continuous attack ∞, 0 or 1, 1 Prior work Srivatsa et al. 2005, TrustGuard §2 model I writes, verbatim: \"Or it could oscillate between building and milking reputation.\" Their model I is a fixed-period square wave , which is exactly our B, R parameterization. We did not sweep models II–IV exponential intervals / random goodness / sinusoidal ramp . Hoffman et al. 2009, ACM Computing Surveys §5.4 list it as a named attack category, oscillation attack , credited to Srivatsa 2005. Their version includes multiple identities taking shifts , which is stronger than our single attacker. Even our headline had been anticipated Withdrawn claims On 2026-08-06 we formally withdrew two kin"},{"t":"The trust on/off experiment on a real model: +8.3%, but statistically nothing happened","u":"/en/blog/e10-paired-null.html","g":"Defense axis","s":"60 paired tasks; switching the trust layer on raised the pass rate from 51.67% to 60.00% — but the 95% interval spans 0, McNemar p=0.332, and statistical power is only 0.163. The same data holds another, more meaningful result.","b":"This was the project's first paired defense experiment on a real model , and the first time we got \"a null result on a real model.\" Design --- --- Model qwen3.6-35b-a3b @ 100.119.113.56:8765 Task bank EvalPlus MBPP+ v0.2.0 378 tasks, sha256 pinned Tasks 60 paired tasks Roster Demo roster, 2 saboteurs planted by hand Pairing key task_id = sha256 prompt + tests , stable across arms Quality check compile_check t.check answer , outside the arm Results Arm Passed Valid Pass rate Time ledger events Calls per pass --- --- --- --- --- --- --- off 31 60 0.5167 6,837.5 s ROUTE 60, ATTEMPT 60, DELIVERED 60 1.94 on 36 60 0.6000 6,168.0 s +REVIEW 180 , AUDIT 60 1.67 The first mismeasurement: reading the wrong field The second outcome variable: more interesting We also measured the amount of work assigned to saboteurs . This is a direct indicator of \"doing bad work and still getting jobs.\" Share assig"},{"t":"Setting the stake too low let attackers earn a 235% return","u":"/en/blog/entry-cost-backfires.html","g":"Defense axis","s":"The most counterintuitive result of the 14 mechanism simulations: free entry blocks completely, yet a stake of 2 gave the attacker a return per unit of cost of 2.354. We deleted two defense designs (an exogenous entry fee and endorser liability) entirely.","b":"On 2026-07-26, we ran 14 mechanism simulations E1–E9, E12–E16 to ask a very ordinary question: How much entry fee should we charge an attacker so that attacking doesn't pay? The answer: don't charge one. Design --- --- Honest residents 5, each with a real Ed25519 identity + a real event chain + a real capability card Attacker 1 Warm-up 8 rounds, so the ecosystem has records that can be endorsed Total rounds 400 Seeds 20 per cell 40 for E9, 30 for E15 Audit rate 0.2 0.05 from E2/E3 on Reviewer accuracy 0.7 chosen by the author, not measured Result: the smaller the stake, the safer Exogenous stake Attacker hits Return Full-block rate --- --- --- --- stake = 0 0.00 — 100% stake = 2 5.00 2.354 0% stake = 5 3.05 0.560 — stake = 10 1.95 0.177 — stake = 20 1.90 0.092 — Why this happens Probation: all the value is in the first step Probation length m Attacker hits Full-block rate --- --- --- m ="},{"t":"The only truly tight constraint in the whole defense system: reviewer accuracy","u":"/en/blog/reviewer-accuracy-binds.html","g":"Defense axis","s":"We swept the stake, probation length, ecosystem size, attacker share, and the correlation of reviewer errors — all of them are slack. The only cell where attacking starts to pay is when reviewer accuracy drops to 0.","b":"In , we deleted the \"exogenous entry fee\" and \"endorser liability.\" The reason was not that they had a weaker effect, but that one was actively harmful and the other was a pure loss. But once those two paths were deleted, what does this system rely on to stop attackers? Five parameters swept, four of them slack Stake Slack. stake=0 blocks completely; stake=2 gives a return of 2.354. Probation length Very tight, but in the opposite direction. m=0 → 6.85, 0% fully blocked; m≥1 → 0.00, 100% fully blocked. One step does it all; after that there is no discrimination at all. Ecosystem size E8 Completely slack. With 2 / 3 / 5 / 10 / 20 honest participants, all 0.00, 100% fully blocked . Attacker share E14 Attackers / total Attacker hits Return per attacker Full-block rate --- --- --- --- 1 / 6 2.30 0.144 5% 2 / 7 4.00 0.125 — 3 / 8 3.80 0.079 — 5 / 10 5.40 0.068 — 8 / 13 7.45 ± 1.07 0.058 0% Co"},{"t":"Zero-config, four batches: three measured no difference, and the fourth vanished when we switched engines","u":"/en/blog/zero-config-measured.html","g":"Gain axis","s":"Zero-config was measured on four batches: DABstep, 711 local cells, 920 Colab tasks and u274. Paid batch: no measured difference for either model; local batch p=0.021 but concentrated in 6 tasks; 920 tasks and u274: no measured difference; v3.7 later measured p=0.0028, but did not replicate on GGUF.","b":"This post records four batches. None of the four looks good, and each looks bad in a different way — that is the point. First, what exactly was measured Zero-config is the v0.9.0 product line. Its goal is not to do a lot; it is to let you keep working as usual once it's installed . No contracts to write, no settings to add, no change of workflow. It does just one thing: at the moment the agent says \"I'm done,\" it sends one message back within the same session. The difference from is not old version versus new. CONFORM owns generation: every candidate is run against the visible acceptance tests, a fail means resampling, up to five candidates, and if none passes it refuses delivery. Zero-config does not own generation: it has no executable signal, no power to retry, and no power to refuse delivery. Put this difference on the table first; only then do the results of the four batches below m"},{"t":"The gate cuts false delivery by about 40%, not in half","u":"/en/blog/gate-cuts-forty-percent.html","g":"Gain axis","s":"An experiment with 836 tasks, 43 blocks and 0 voided. The thesis's \"halved\" was wrong — because the two rates used different denominators. This correction changes the gate's effect from \"in half\" to \"roughly 40%.\"","b":"What false delivery is Term Definition --- --- Delivery success The system actually lets it through, and the saved candidate passes the hidden evaluation False delivery The system lets it through, but the candidate does not pass the hidden evaluation So there are three possible ways to count a \"false delivery rate.\" Each is correct, but they give different numbers. The R532 data --- --- Model qwen/qwen3.8-27b Q4_K_M, non-reasoning Tasks 836 Blocks 43 Completion 43 / 43 Data rows 2,508 Voided 0 V/GT fingerprint 199,019 items CLEAN Receipt chains 86 chains / 3,895 entries, all verify This is the experiment with the cleanest data quality in the project — 0 voided, 0 V/GT violations, all 86 chains verify. Three ways of counting Arm Accepted False deliveries False deliveries / all 836 False deliveries / accepted --- --- --- --- --- CONFORM 811 120 14.4% 14.8% OFF 625 211 25.2% 33.8% The corre"},{"t":"This is a success-rate gain, not a token-efficiency gain","u":"/en/blog/token-cost-ledger.html","g":"Gain axis","s":"Recomputed call by call, 197 cells, zero differences. H-MIX uses 2.23–2.78× the tokens of a single draw, and in four of the five replications it was more expensive per correct delivery than the gate. \"The same ceiling\" is not \"the same actual cost.\"","b":"Two definitions that are easy to confuse --- --- reasoning A subset of completion , not an extra amount added on top Failed requests The 116 failed requests in the core experiment have no usage and are counted as unknown not filled in as 0 R460's cost Arm Delivery rate tokens / task tokens / correct delivery --- --- --- --- OFF 58.33% 2,913 4,994 CONFORM 70.83% 5,805 8,195 OFF5 65.00% 15,004 23,084 H-PI 75.83% 9,448 12,459 H-OC 80.00% 7,432 9,290 H-MIX 84.17% 5,677 6,745 Three ratios Comparison Per task Per correct delivery --- --- --- H-MIX vs OFF 2.23 – 2.78× 1.65 – 2.11× H-MIX vs CONFORM 0.92 – 1.56× R460R / 1.07 – 1.69× R529 — OFF5 vs CONFORM 2.16 – 2.85× , and a lower delivery rate every time — \"Without spending more tokens\" does not hold Our original headline claim said the revision loop, compared with resampling with a fresh attempt, \" delivers more, without spending more tokens ."},{"t":"After five replications: the headline claim lost to its own stability rule","u":"/en/blog/five-replications.html","g":"Gain axis","s":"R460 measured the revision loop at 13.33 percentage points above resampling with a fresh attempt, Holm p = .0112. After five same-task replications, all five were not significant, 0/5 passed correction, and the status was downgraded to \"Same sign, unresolved.\"","b":"The order of events 2026-09-07 : R460's preregistration fixed four state rules. 2026-09-11 : R460 finished running LCB v2, 120 tasks, 6 blocks × 20 . Arm Successes Rate --- --- --- OFF 70 / 120 58.33% CONFORM 85 / 120 70.83% OFF5 78 / 120 65.00% H-PI 91 / 120 75.83% H-OC 96 / 120 80.00% H-MIX 101 / 120 84.17% H-MIX − CONFORM: b=22, c=6, +13.33 pp , conditional interval 4.22, 19.46 , Holm p = .0112 → EFFECTIVE . 2026-09-11 : R460R's preregistration fixed the claim rule for \"five same-task replications.\" 2026-09-12 : all five runs finished. The preregistered claim rule --- --- Judge each run separately All five must pass Forbidden Pooling n, averaging, picking one run to report Decision threshold Conditional interval lower bound 0 and passes Holm correction Results of the five replications Delivery rate % : Arm r1 r2 r3 r4 r5 --- --- --- --- --- --- OFF 57.50 54.17 57.50 51.67 51.26 OFF5 6"},{"t":"The only thing that held stably across five replications: the loop beats a single draw by 17 to 29 percentage points","u":"/en/blog/loop-vs-single-shot.html","g":"Gain axis","s":"All 15 blocks pass their own Holm correction, with differences from +17.5 to +29.2 percentage points, across five same-task replications and four mutually exclusive task sets. This is the hardest result in the whole project.","b":"The difference in what is compared is the whole point Comparison Result Verdict --- --- --- Loop vs a single draw OFF +17.5 ~ +29.2 pp , 15/15 pass Holm Held Gate vs a single draw OFF +14.17 ~ +19.97 pp Held Loop vs resampling with a fresh attempt CONFORM +0.83 ~ +5.83 pp, 0/5 pass Same sign, unresolved Loop vs voting OFF5, exploratory −1.74 ~ −15.83 pp Same sign, unresolved The 15 blocks in detail Result --- --- Number of blocks 5 replications × 3 H arms = 15 Blocks passing their own Holm correction 15 / 15 Δ range +17.5 ~ +29.2 pp Gate vs a single draw CONFORM − OFF, five runs: r1 r2 r3 r4 r5 --- --- --- --- --- --- CONFORM 86 87 88 87 85 OFF 69 65 66 60 58 Δ +14.17 +18.33 +17.50 +19.17 +18.97 Uncorrected p 0.0015 — — — — Attribution: the bulk of the gain is gate + resampling Stacking the five runs' data together: Difference --- --- Three H arms vs OFF +17.5 ~ +29.2 pp CONFORM vs OFF +"},{"t":"The gain comes from the loop, not from the first-round prompt","u":"/en/blog/loop-not-prompt.html","g":"Gain axis","s":"We pulled out the complete first-round responses and re-scored them offline: the loop contributes +19.17 / +17.50 percentage points, the first-round prompt only +6.67 / +8.33. 21 tasks were rescued by the loop.","b":"The first round of the three arms The first-round prompts of H-MIX, H-PI and H-OC are almost the same: the original task text, the list of standard libraries allowed to be imported, and one line saying \"first check your function by hand against the examples in the task.\" The difference is in what happens afterwards : Arm After a failure --- --- H-MIX Compute the failure signature and paste the failure text back to the same worker H-PI The original pi-style feedback loop H-OC Plan → diagnose → self-test Results: split into two effects Arm Loop effect final − first round Prompt effect first round − OFF Tasks rescued by the loop First-round visible pass rate --- --- --- --- --- H-MIX +19.17 / +17.50 pp +6.67 / +8.33 pp 21 74.2% H-PI +18.33 / +15.83 pp −0.83 / +1.67 pp — 71.7% H-OC +19.17 / +18.33 pp — — — Why this design could do it {{note:method Three design choices made the attribution po"},{"t":"The MDE is 4.31 percentage points, but the task bank has only 378 tasks","u":"/en/blog/mde-and-power.html","g":"Gain axis","s":"For the comparison of the gate against majority vote, we can calculate that 491 pairs are needed for 80% statistical power, but the task bank actually has only 378 tasks. This \"Same sign, unresolved\" is not unfinished work; it is physically unmeasurable.","b":"Calculate first, then run Before R529 cross-bank , we first calculated one thing: how many tasks do we need to have an 80% chance of detecting a real effect when one exists? --- --- MDE @ n = 371 4.31 percentage points Needed for 80% statistical power about 491 pairs Actual task bank size 378 tasks Measured results Task bank n Δ Statistics --- --- --- --- LCB v2 120 +6.67 pp p = 0.15 LCB v3 189 +1.59 pp 95% CI −3.17, +6.35 , p = 0.6636 MBPP+ clean replication new 192 tasks 192 +4.69 pp 95% CI −1.11, 9.42 The widths of the three intervals Task bank 95% CI Half-width --- --- --- LCB v3 n=189 −3.17, +6.35 4.76 pp MBPP+ replication n=192 −1.11, +9.42 5.27 pp MBPP+ pooled n=371 — Half-width corresponds to the MDE of 4.31 pp What the same concept looks like in other experiments Experiment n Statistical power Needed for 80% --- --- --- --- E10 trust switch 60 0.163 337 R460 for +10 pp 120 0.43 "},{"t":"The harder the task, the more the acceptance gate is worth","u":"/en/blog/harder-tasks-more-worth.html","g":"Gain axis","s":"The early-stop gate delivers 7.8 percentage points more on MBPP+, and 20.9 percentage points more on hard LiveCodeBench problems. And the \"leak rate\" drops from 44 to 11 — refusing delivery is worth the most on hard tasks.","b":"A comparison along three dimensions MBPP+ LCB v1 hard tasks --- --- --- Tasks 179 91 OFF single draw failure rate 31.8% 48.4% Gate vs single draw +7.82 pp +20.88 pp b / c rescued by early stop / right on the single draw 14 / 0 19 / 0 Candidate-pool ceiling 82.68% 75.82% Single-draw success rate 78.2% 45.05% Leak rate: refusing delivery is worth the most on hard tasks Arm MBPP+ leaks LCB hard-task leaks --- --- --- OFF single draw — 44 / 91 OFF5 five-way vote — 35 CONFORM gate — 11 MBPP+ LCB hard tasks --- --- --- Gate vs five-way vote +4.47 pp +4.40 pp p 0.0963 0.424 Gate vs single draw +7.82 pp +20.88 pp p 0.0001 < 0.0001 This is because on hard tasks the model is more likely to confidently hand over something bad . The gate's mechanism is \"run your tests, deliver only on a pass\" — on hard tasks it can still judge \"this doesn't pass,\" while a voting mechanism can only be wrong when all "},{"t":"The birth of the judging gate: execution instead of opinion, receipts instead of votes","u":"/en/blog/conformance-gate.html","g":"Gain axis","s":"We deleted the AI review committee and replaced it with \"run the customer's acceptance tests, deliver the first candidate that passes, refuse delivery if none passes.\" On three task banks it delivered 4.58 to 19.17 percentage points more, at only 1.5 calls per task.","b":"The rule frozen word for word 1. Generate candidates in order. 2. Run the visible acceptance tests on every candidate in a sandbox. 3. Let the first candidate that passes through, and stop generating. 4. At most five. If none passes, refuse delivery . 5. Refused delivery counts as a failure, and the denominator is all tasks. Why \"stop at the first pass\" works Because the ceiling of any selection rule is the share of tasks where at least one candidate in the pool is correct . Majority vote can't escape this ceiling either. Two saved batches of five-candidate data: Task set Candidate-pool ceiling Tasks with all five wrong --- --- --- g_r441 82.68% 31 / 179 g_r356 85.03% 22 / 147 {{note:method This is a very hard ceiling Assuming each candidate is generated independently, the probability of \"at least one is right\" is the ceiling. No number of extra calls and no selection rule can break thro"},{"t":"Only after our own experiments excluded two explanations did the mechanism have room to change","u":"/en/blog/hypotheses-excluded.html","g":"Gain axis","s":"\"The model is too strong\" and \"the tasks are too easy\" were two explanations proposed by humans. Two experiments excluded both, and only then was the reviewer mechanism forced into a redesign — and the direction was to change the referee, not the reviewer model.","b":"The situation ON vs OFF5: 75.25% vs 77.23%, a difference of −1.98 pp , p = 0.7905. H-A: the ceiling hypothesis proposed by a human on 2026-09-01 qwen3.6-35b is too strong. So every model gets it right, and the mechanism has no chance to make a difference. Switch to a weaker model and the gain will show. Experiment R440N / E1 --- --- Design Single model gemma-4-12b, 179 tasks Audit Independent audit recomputed row by row, fully consistent with R516 Audit receipts rows.jsonl 525 lines, sha 440d973c ; calls 1,986, sha 3d24b73d Arm n Passed Failure rate --- --- --- --- OFF 179 122 31.8% Wilson 25.5, 39.0 ON 167 +12 void = 6.7% 122 26.9% OFF5 179 132 26.3% Comparison Difference Statistics --- --- --- ON vs OFF5 equal budget −0.60 pp b=11 c=12, p = 1.0000 ON vs OFF +5.99 pp b=17 c=7, p = 0.0639 OFF5 vs OFF +5.59 pp b=12 c=2, p = 0.0129 Reviewer grounded lift +4.19 pp 95% 0.00, +8.58 H-A exclud"},{"t":"The reviewers track task difficulty, not whether the answer is right","u":"/en/blog/reviewer-tracks-difficulty.html","g":"Gain axis","s":"The same reviewer layer judges almost everything as pass on MBPP+ (0.305 error rate) and almost everything as fail on LiveCodeBench hard tasks (0.46 error rate). It follows a global property, not this particular answer.","b":"Two task banks, the same reviewers MBPP+ LCB v1 medium + hard --- --- --- Tasks 179 91 OFF-arm failure rate 31.8% 48.4% Actual error rate of first drafts 0.305 0.460 Reviewers' raw FAIL claims Few almost always PASS 234 / 261 = 89.7% almost always FAIL Reviewer layer − zero-intelligence baseline −2.99 pp +0.77 pp A surprise during the audit While checking the FAIL claims, we found a large number of \"discarded FAIL claims\": --- --- Raw FAIL-claim rate in round346 1.2% – 25.8% Of those, discarded as outside_input_contract 73.6% – 100% Another observation: a collective blind spot Pooling the three runs, on the 49 tasks that really do fail: --- --- Total tasks 275 True negatives the TN = 0 part 0 Votes cast by the 3 reviewer agents 147 Of which said \"pass\" 145 98.6% Tasks that ever got a fail vote 2 / 49 and both times outvoted by the majority {{note:warn On the tasks that go wrong, the revi"},{"t":"\"The hidden tests are lossless\" is empty by construction","u":"/en/blog/v-gt-is-not-lossless.html","g":"Gain axis","s":"A criterion R440P set in advance was refuted nine days later: in LCB, 91/91 hidden test sets are supersets of the visible tests, so \"would pass hidden but fails visible\" cannot happen by construction. This criterion had zero statistical power.","b":"The original criterion Our gate uses only the visible tests to decide what to let through. So one thing had to be verified: There will be no candidate that \"would pass the hidden tests but failed visible acceptance.\" If there were, it would mean the visible tests miss some requirements, and the gate would let bad things through. Results on six datasets Dataset Candidates Passed hidden but failed visible --- --- --- g_r441 MBPP+ 895 0 g_r356 MBPP+ pool 735 0 g_r443 LCB v1 replay 455 0 g_r447 LCB v2 real run 120 0 g_r461 LCB v3 real run 189 0 Total 2,394 0 But R440Q found out why it is 0 So the criterion \"visible filtering is lossless\" has zero statistical power on these two task banks. 0/2394 does not mean \"we verified that it is lossless\"; it means \"the definition of this task bank guarantees it holds.\" Three stricter ways of putting it R440Q upgraded it into a main conclusion: If a requ"},{"t":"Our AI review committee lost to \"generate five times, take the majority\"","u":"/en/blog/committee-loses-to-majority.html","g":"Gain axis","s":"Same 5 calls, same set of tasks: a committee of three AI reviewers plus one revision lost to \"generate five times and take a majority vote.\" The point estimate of the difference was −1.98 percentage points — and after we fixed an unrelated bug, it became +2.97.","b":"On 2026-08-30 we ran the most critical experiment of the whole project. The result: the core mechanism we designed lost . Background: the committee design Our architecture originally looked like this: The problem: the referee of this whole thing is an AI . And our control was the most brainless kind: generate the same task five times and take a majority vote over the five . Under the same call budget 5 calls each , which one has the higher delivery rate? Design --- --- Task bank MBPP+ EvalPlus v0.2.0 Tasks 179 tasks, 6 blocks Three arms OFF single sample / ON full mechanism / OFF5 five tries, majority Criteria frozen 2026-08-24 and 2026-08-28 Statistics Paired McNemar Results Criterion Verdict --- --- ① The gauge has signal ✓ 26.44% Wilson 20.4, 33.4 ② The three arms differ discernibly Not measured ③ ON beats OFF5 ✓ The answer is no Paired results n = 101, no arm void : Measure ON OFF5 D"},{"t":"The measuring tool got it wrong seven times, and every error leaned against itself","u":"/en/blog/measuring-measurement-errors.html","g":"Records & multi-party execution","s":"The A/B harness for the interactive TUI had seven measurement errors, plus two more that exist only in code comments. Every one was caught on a real pi, and every one pushed the numbers to look better than they really were — 8 fabricated results, and 5 cells whose prompts were never sent.","b":"This post records the measuring tool's own errors. The first sentence of commit b0eb0680 is explicit: \"This is not cosmetic formatting. Every one was caught on the TUI of a real pi 0.87.1, and every one points in the direction of making the harness unfavorable to itself or favorable to the agent — in other words, these numbers would otherwise have looked better than they really were .\" original: 「這不是排版修飾。每一條都在真 pi 0.87.1 的 TUI 上抓到，而且每一條的方向都是讓 harness 對自己不利或對 agent 有利，也就是說這些數字原本會長得比實際好看。」 \"The same direction\" is the key. If the seven errors had pointed in random directions, that would be noise; if they all point the same way, it is systematic — meaning this harness has a structural problem. --- Why an interactive TUI is especially hard to measure An interactive interface has no API to call. Everything to be measured where the gate banner appears, which turn the model has reached, whether "},{"t":"An attacker re-mints the whole chain with their own key, and the receipt viewer still says it passes","u":"/en/blog/offline-viewer-limit.html","g":"Records & multi-party execution","s":"194 receipt chains, 9,841 entries, all verify. But the offline single-file viewer has a structural limit: if the whole chain is re-minted once, it still passes. This is not a vulnerability; it is the definitional boundary of a signature.","b":"Three things chain verification can't catch Can't catch Why --- --- Truncation The chain has no trusted chain head. Cut off the tail and it still passes. Forks If the same party signs two chains with different content for the same event, both verify. Omissions For an event that was never written down, a receipt cannot claim to be one of its entries. R455: the limit of the offline viewer We built a single-file offline viewer with zero external resources and zero network requests . The receipts are embedded in the file, and opening it directly via file:// runs the full chain verification: File Size --- --- exhibit/receipts/receipt_viewer.html 0.3 MB exhibit/receipts/receipt_viewer_multiparty.html 4.5 MB Then we attacked it: Why this is not a vulnerability that can be fixed This sentence has a sharper version over in multi-party execution: A signature identifies a key, not a principal. A ma"},{"t":"The lying key was named all 273 times; honest keys were falsely accused 0 times","u":"/en/blog/named-dissent.html","g":"Records & multi-party execution","s":"k=3, 1 controlled lying key, 5,519 testimonies. Lying cells named 273/273, honest keys falsely accused 0/1,840, contradictory-signature cells ruled correctly 58/58. How strict these numbers are depends on whether you know the mathematical upper bound ⌊(k−1)/2⌋.","b":"Three things 1. Execute and attest : each executor runs acceptance with its own renderer and sandbox, and signs into its own event chain the hashes of the task, the candidate, the suite and the rendered result, along with the visible acceptance result. 2. Tally : first check whether the suite passed the gauge, then verify the testimony one entry at a time . The same executor signing two testimonies with different content for the same event → proof-grade fault , voided. 3. Vote and name : decided by majority. Votes short of a quorum, or a tie → undecided , and the shipping rule treats undecided as not passed. The minority is listed by name , with its event hash and signature attached. R454's results k = 3, with 1 controlled lying key K3 : Check Result --- --- Lying cells that named K3 273 / 273 Honest keys falsely accused 0 / 1,840 Contradictory-signature cells ruled correctly 58 / 58 Tot"},{"t":"After turning the acceptance suite from code into data, the delivery-rate drop went from 75% to 2.7%","u":"/en/blog/suite-as-data.html","g":"Records & multi-party execution","s":"An attacker wrote a stateful variant of the gauge: the gauge scored perfectly in both directions, yet the delivery rate dropped 75 percentage points. The fix was to leave the suite with only three things: a dialect, an entry-point name, and a literal list of input — expected value pairs.","b":"The problem: the gauge can be fed into uselessness 2026-09-06, R451. Stage Result --- --- Suite bound into the commit, plain suite counts as passing if it loads 371 / 371 all refused — normal Stateful gauge variant Delivery rate drops 75.20 pp What the gauge reported ok perfect in both directions How the attack works The original suite was a piece of trusted code . It could: 1. Read external state. 2. Change itself or the environment when called. 3. Behave correctly only on the \"known wrong samples.\" Multi-party execution does not defend against this — because every vote is honest, and every chain verifies . The fix: SuiteSpec Restrict the suite to three things : --- --- 1. Dialect Python / a specific version 2. Entry-point name solve 3. A literal list of \"input — expected value\" pairs Pure data The executor runs only its own deterministic renderer to generate the test code from this dat"},{"t":"The definition changed five times: from \"a responsibility structure\" to \"no delivery without a receipt is accepted\"","u":"/en/blog/definition-five-versions.html","g":"Project history","s":"Over nine months, \"what Vacant is\" was rewritten five times, and each time the claim got smaller: from \"a responsibility structure agents adopt voluntarily\" to \"a delivery without a verifiable receipt is not accepted.\" The shrinking process itself is this project's contribution.","b":"The five versions When Definition Why it changed --- --- --- 2026-04–05 A \" resident form \": a responsibility structure that agents adopt voluntarily, with keys, records and public reputation; adjudicated by the network itself, with no central referee The starting point. Theory documents V3–V5 went through two rounds of adversarial review; 38 attack vectors were each marked as prevents / detects / raises the cost 2026-06 \" A persistent, evolvable, accountable subagent ,\" then narrowed to a capability layer that agents can call on demand, with verify–revise at its core The consensus meeting's critique by 63 agents caught fabricated citations and unimplemented mechanisms; \"one instance per agent\" cost too much compute 2026-07 Resident = body a frozen model + memory an event stream ; responsibility lives in reviewed memory KS-1 refuted the single-agent responsibility hypothesis; the indepen"},{"t":"KS-1: On a frozen model, \"writing the consequences to look real\" and \"making the consequences real\" are the same thing","u":"/en/blog/ks-one.html","g":"Project history","s":"On 2026-06-28, the two main validation designs were killed by adversarial workflows of 17 and 10 agents. The fact that killed them later became the methodological core of the whole project.","b":"The two designs that were killed The main validation of 2026-06-27/28 set out to test \"whether a responsibility structure can change an agent's behavior.\" --- --- Version one \"Main validation redesign: responsibility structure on decaying tasks\" original: 「主驗證重設計：責任結構在衰減任務」 Version two \"Main validation, final v2 post-adversarial \" original: 「主驗證定版 v2（post-adversarial）」 Adversarial workflow 17 agents / 10 agents Result Both killed The voids are kept as design history — not deleted, but marked \"this is design history, not a working mechanism.\" The sentence that killed them Why this is fatal The logic by which KS-1 holds is simple: 1. The model is frozen no fine-tuning, no learning . 2. The only input channel that can influence it is the prompt . 3. So any \"mechanism\" that is to influence behavior must go through the prompt. 4. Therefore, in this setting, a \"mechanism\" and \"a better prompt\""}]}