VacantAn accountability layer for AIGitHub

Experiments · Results / records ·

Records & multi-party execution

194 receipt chains and all 9,841 records verify; the lying key was named 273/273, and honest participants were falsely accused 0/1,840. But chain verification cannot catch truncation, forks, or omissions, and mutual execution has no defense at all against the acceptance suite itself being corrupted.

Records & multi-party: verdicts of 3 claims

Held3/3

#Receipt chain verification

BatchChainsRecordsResult
R460R (five replications)—6,674All passed
R529 (four sets across task banks)—3,167All passed
Total1949,841All passed independent signature verification
R532 (larger model)863,895All passed
ABPI audit195—All passed
R454 (multi-party execution)—5,519 testimonies0 signature-verification failures
E10 (after the channel-separation revision)—180/180 reviewer envelopesStill verifiable

Three things are verified: sequence numbers are contiguous, each record's previous hash matches, and the Ed25519 signature is valid. Coverage is only CONFORM and the three H arms — the OFF and OFF5 arms have no receipts.

#V/GT leakage audit

VersionScopeResult
v1—90 false positives
v2R460R 30 blocks + R529 37 blocksZero violations
v2 independent procedure2,497 submission texts, 164,963 comparison itemsZero hits (with a positive control)
Retroactive back-scan of all arms (2026-09-18)179 records, roughly 349 × 10⁴ comparison items165 no violation, 10 unverifiable, 4 judged violations

All 4 violations belong to R530, which was never closed out. The breakdown for the first three runs of R460R: 95,090 comparison items, 57,248 checked, 37,842 skipped (39.8%).

#Multi-party execution: run each other's work, don't review it

Liars named 273/273

#Simulation sweep (200 cells)

FindingNumber
At a fixed corruption ratio, the delivery rate does not change by a single digit as k goes from 1 to 7collusion 67.39%, sabotage 0%
k=3 / quorum=2, one vote missing1-1 tie → undecided, no one named

#Real runs

ExperimentScaleResult
R453 cross-machine consistency368 tasks × 5 candidates = 1,840 candidates, macOS + Linuxvisible acceptance labels 1,840 / 1,840 agree; both chains verify
R454 namingk=3, 1 lying keylying cells named 273 / 273; honest participants falsely accused 0 / 1,840; cells with contradictory signatures ruled correctly 58 / 58; 0 signature failures across 5,519 testimonies

#Mathematical upper bound

Tolerance bound⌊(k − 1)/2⌋
False-accusation rate once past a majority0.175 (r446 371 tasks × 5) / 0.374 (r443 91 tasks × 5)

When a majority colludes, the result flips: honest participants get named instead, and the mechanism itself does not know which side of the threshold it is on.

#The fixed point: the acceptance suite itself is corrupted

If the suite is replaced with one that "passes on load," then every vote is honest, every chain verifies, and the system is still delivering garbage.

StageChange in delivery rate
Suite bound into the commit, trivial suite371 / 371 all refused (normal)
Stateful variant−75.20 pp, while the gauge reports ok
Fix: SuiteSpec (exec smuggling)368 / 371 → 0 / 371
Residual: generic_gauged+2.72 pp
Post-hoc upper bound: weak_oracle+4.35 pp

After turning the acceptance suite from code into data

#The offline viewer's structural limit

R455: the single-file offline viewer (exhibit/receipts/receipt_viewer.html, 0.3 MB; multi-party version 4.5 MB) embeds the receipts inside the file, with zero external resources and zero network requests.

The attack: re-mint the whole chain with the attacker's own key, edit the embedded receipts to match, and it still verifies.

The mitigation is to print the three vacant_ids on a physical label, so that outside the chain there is an anchor that cannot be re-minted along with it.

#Field tests of hooking into existing agent platforms

#Five agents

AgentVersion
pi0.85.1
OpenCode1.18.31
Claude Code2.1.278
Codex0.147.0
Hermes0.19.0

20 cells (2 cells × 2 runs × 5 agents) all met the criteria written in advance: 20 receipt chains / 40 records.

#Public task bank gate

LCB v2 × 10 tasks.

BatchResult
pbgate (20 cells)delivered:rejected = 15:5
pbgate2 (30 cells, 1003, reasoning on)pi 5:5, Claude Code 7:3, OpenCode 7:3; hidden checks 19/19; stub negative control 10/10
pbgate3 (30 cells, 1004)24:6; hidden checks 24/24
How far the three assistants agreeThey agree on only 4 / 10 tasks

Compared with pbgate2, the three agents flipped on 5 / 2 / 2 cells.

#Hermes and MCP end to end

StageResult
2026-06-19 observation2 calls locally; 16 consecutive verify_fix calls on the VM, all verified:true + signed
2026-07-05 end-to-end recordHermes autonomously chose delegate; 2.6 seconds per round, 1 model call, about 20 seconds end to end; 179 tests all green; credit 0.5 → 0.535
2026-07-09 feasibility checkafter a VM reset, pytest 189/189; 24/24 delegate, 96 events; all 4 bad deliveries in the on arm were caught, 3 bad deliveries in the off arm passed silently, leaving zero accountability trail; replay 29/29

#pi-tty

40 cells, 5 arms: accepted PRT 6/10, PTP 5/10, INT 6/10, INT2 3/5, INTQ 0/5. The first-request bytes of -p and the TUI were identical 5/5.

#R535 retry channel

gemma-4-12b + pi 0.85.1; a self-built set of 90 tasks (S1 50, S2 40) × 4 arms = 360 cells.

S1 (retest after H1 was refuted on M1)+0.860 [+0.760, +0.940], b=43 c=0, p = 2.27e-13
S2+0.400 [+0.225, +0.550], b=17 c=1, p = 1.45e-4 → CONFIRMED_POSITIVE

#Negative audits

#ABPI audit (391 cells, 2026-09-20)

CheckResult
Any gaps in the server-side conversationNone; all 2,755 turns match
Receipt chain195 / 195 verify
Receipt gradeAll grade C (no sandbox, no hooks)
Does "blocked at the exit" holdIt does not

Refused delivery is only a flag and an exit code, and the outer script does not read the exit code: in 7 of the 59 refused cells, the files were copied out and scored. The real upstream URL was still left in the agent's environment variables (one command could bypass the proxy; it just didn't happen in that batch). requests_seen is a lower bound. The hooks were never installed.

Receipts

#Digital twin exhibit (54 cells, 630 real model calls)

ConditionAccepted
Interface does not provide the necessary information0 / 27
Interface provides the necessary information24 / 27
Rescued by the retry loop0 / 30

There were also 82 cases where the agent declared success but the visible check failed, plus 8 timeouts that were killed.

#2026-07-26 exhibit (TWIN)

54 cells: 30 stopped at the gate, 0 rescued by retrying (in 27 of them, the task deliberately hid the function name). Under the default settings, refused delivery is a flag, not a wall; to actually block the files, it has to run inside the sandbox.

#Framework hooks and the wall (2026-09-20)

Negative controls 7/7; bwrap blocks everything except the gate; the hooks in all five agents can intercept shell commands — but an agent can delete its own hook configuration.

The first grade-A receipt from a real agent: pi running on 1003, with 0 unexplained calls.

#BARE arm (claiming success empty-handed)

b1003: 25 cells; b1004: 0 cells.

#Measurement tools for the integration layer

agentlane (2026-09-22, commit 36609614): a cell passes only when three independent sources agree — requests_seen ≥ 1, the line count of the proxy server's index == requests_seen, and the agent's exit code has a value. Every round runs a negative control (upstream pointed at a closed port). Whether the delivery is accepted is deliberately not a criterion.

The first smoke-test round caught this: OpenCode ignores the specified working directory and writes the delivery somewhere else, so mediation passed across the board but every cell was refused delivery. In addition, OPENCODE_CONFIG_CONTENT wipes the user's MCP settings.

#Next

Last updated: · about 5 min read · this page cites 2 adjudicated claims

Shih Hsin UniversityDepartment of Information and Communications, Shih Hsin UniversityVacant

Department of Information and Communications, Shih Hsin University · 2026 capstone project · on-site exhibition