VacantAn accountability layer for AIGitHub

Start · Quickstart / ten minutes ·

Quickstart

Walk the whole path with one task that has a known correct answer: write the requirement and its acceptance tests, let the AI assistant do the work, see how the gate rules, learn what each field on the receipt contains, and which three things chain verification cannot catch.

#You need to prepare two things

The gate runs your tests; it does not ask a model "does this look right." So:

  1. A requirement — say clearly what you want.
  2. Acceptance tests that can run — what the input is and what the expected output is.

#What a task looks like

The acceptance tests are split in two halves:

Who can see itPurpose
Visible VThe agent can read itThe system uses it to decide whether to let the work through or refuse delivery; the revision process also gets only its failure messages
Hidden GTOnly the after-the-fact scoring can see itMust not enter generation, selection, revision prompts or stopping conditions

A program that passes visible but fails hidden is a false delivery. This is the operational form of the problem "the output looks like the requirement but isn't the requirement."

# suitespec.py — the acceptance suite is "data," not code
# a suite contains only: dialect, entry-point name, a literal list of input—expected value pairs
# the executor only runs test code generated from this data by its own deterministic renderer

Why "V ⊆ GT" empties the word "lossless" Here is a pitfall we hit ourselves that is worth knowing up front: in the two task banks we used, the hidden tests were always a superset of the visible tests, so "passes hidden but fails visible" is impossible by construction. This finding refuted one of the premises in our own preregistration.

#Walk through it

# 1) start mediation, pointed at your model (LM Studio / Ollama / cloud API all work)
vacant run --agent opencode --base http://localhost:1234 --model your-model

# 2) the agent finishes and the process exits → this is the delivery point
#    the system runs the visible acceptance tests on the frozen workspace snapshot

# 3) read the verdict
#    0 = delivered 20 = refused delivery 22 = infrastructure failure

Along the way you get two layers of output:

Action tickets (one per task) — the requirement, what the AI said, which acceptance tests were run, and what the verdict was.

Receipts (one entry per attempt + one verdict entry per task) — see below.

#Reading a receipt

Every cell on the receipt is signed and locks the hash of the previous cell. So:

  • Change one character → the chain breaks at that cell.
  • Anyone holding the public key can recompute it offline, with no need to trust us.

How to verify:

vacant audit <name>     # re-verify the signature chain of a logbook (PASS / FAIL)

The offline single-file viewer we keep ourselves (exhibit/receipts/receipt_viewer.html, 0.3 MB; the multi-party version 4.5 MB) embeds the receipts in the file, with zero external resources and zero network requests; saved and opened directly via file://, it still runs the full chain verification.

#Measured results on five agents

On a real model, all 20 cells met the criteria written down in advance (20 receipt chains / 40 entries):

AgentVersion
pi0.85.1
OpenCode1.18.31
Claude Code2.1.278
Codex0.147.0
Hermes0.19.0

#Next

Last updated: · about 2 min read

Shih Hsin UniversityDepartment of Information and Communications, Shih Hsin UniversityVacant

Department of Information and Communications, Shih Hsin University · 2026 capstone project · on-site exhibition