<!-- normative for harness; the review decision is the acceptance condition -->

# A2 oracle-consolidation review brief — R08 re-review acceptance

Status: **closed as no change** (2026-09-30), through the re-scoping option
below, after re-measurement against the Phase 3 oracle. This brief was written
as the acceptance vehicle for performance-plan A2 ("Oracle consolidation"): the
item is deliberately **not implemented**, because its change re-opens the R08
evaluator-isolation GO verdict — the per-check disposable-copy design is the
basis of that verdict, and consolidating it would require an adversarial
re-review as acceptance, not an assumed guardrail
([performance plan](performance-plan.md#a2-oracle-consolidation)).

## What A2 proposed, and what the measurement now says

A2 proposed reducing the hidden oracle's per-check overhead — "one sandbox per
candidate and one read-only copy (or copy-on-write overlay) instead of
`copytree` + digest + fresh sandbox per check".

First measurement (2026-09, frozen harness, four-check hidden suite, known-good
tree):

| Component | Cost | Share |
|---|---|---|
| Full grade (4 checks: 4 × copytree + 8 snapshots + 4 sandbox runs) | **0.81 s** | 100 % |
| 4 × (copytree + before/after digest) | **~0.00 s** | ~0 % |
| 4 sandbox runs (setup + workload) | **~0.81 s** | ~100 % |
| A4 one-sandbox-per-attempt saving already captured (commands) | n/a | — |

The `copytree` + digest half of the proposal is **worth nothing** on the
measured tree: the disposable copies are four small files and the digests are
cheap. The only meaningful saving left is **sharing one sandbox across the four
checks** (≈ 3 × 80 ms setup ≈ 0.24 s per grade, ~30 % of grading), and that
saving is exactly the part that removes per-check isolation.

This measurement is why the item is not implemented on the implementer's own
initiative: the win is small (and smaller still now that A1 caches decisive
grades across identical trees), while the design change is precisely what the
R08 GO rests on.

## Second measurement: against the Phase 3 oracle (2026-09-30)

The first measurement predates the project-owned oracle, so its cost model no
longer describes the grading that actually runs. Re-measured per check on an
intact real-repository tree, one check graded at a time:

| Check | Cost | Share of a grade |
|---|---|---|
| `gate-check-tests` (frozen suite, 516 tests, in-sandbox) | **13.96 s** | 91 % |
| `gate-check-static` | 0.22 s | 1 % |
| `registry-counterexample-floors` | 0.39 s | 2 % |
| `registry-open-defect-floors` | 0.30 s | 2 % |
| `registry-guard-floors` | 0.20 s | 1 % |
| `registry-fidelity-floors` | 0.24 s | 2 % |
| manifest build (once, not per grade) | 2.24 s | — |

A grade is **~15.3 s**, and both of A2's halves are now measurably irrelevant
to it:

- **The copy half is still worth nothing.** `gate-check-static` performs two
  full-tree copies of the *real* repository plus a sandbox setup in 0.22 s
  total, so the per-check copies and digests remain negligible. The first
  measurement's premise survives the move to a real target.
- **The sandbox half is worth ~0.25 s of 15.3 s, about 1.6 %.** Five saved
  setups at ~50 ms each is the entire remaining win, and it is precisely the
  part that removes per-check isolation.

A2's target is therefore not where the time is. The cost is one check running
the target's real test suite in-sandbox per candidate, and A2 does not touch
it. A1's decisive-grade caching does not either, because every distinct
candidate tree is a cache miss — so over *N* candidates this is ~14 s × N.

**Closing rationale.** The item is closed as *measured, no change*: the win is
~1.6 % of a grade, and the price is an adversarial re-review of a security
verdict whose subject has since changed shape. Phase 3 replaced the per-check
execution shape this brief reasons about — overlay mounts, a declared-scratch
execution tree, a second candidate copy, and identity checks moved after the
probe — so a review launched now would adjudicate a different design than the
one option 1 and option 2 were written against, and would plausibly have to run
twice. The R08 verdict is left untouched while the oracle's shape is still
moving.

The real cost is recorded here rather than solved: narrowing
`gate-check-tests` (running the gate once per manifest when candidate source is
unchanged, or a smaller frozen corpus) is a different item with its own review
surface, not an isolation change.

## What the consolidation would change (the decision before the review)

1. **One sandbox per candidate, per-check child processes** (the A4 sequencer
   pattern applied to the four hidden drivers). Each check's driver runs as a
   child of one confined process; the trusted sequencer hashes each check's
   output. *Open question for the review*: check 2 can observe filesystem state
   check 1 left behind in the shared writable layer — per-check disposable
   copies currently make that impossible. Either the sequencer must reset the
   writable layer between checks (restoring equivalence), or the review must
   rule that driver-level isolation is superseded.
2. **One read-only copy + per-check writable layers** (copy-on-write). A
   read-only graded tree *supersedes* tamper detection (a write fails with
   `EROFS` instead of being detected by the before/after digest). The review
   must decide whether "mutation becomes impossible" is an acceptable
   supersession of "mutation is detected", including for candidates that write
   bytecode caches or temp files into their own tree.
3. **Keep the per-check disposable copy; drop only the redundant digest**
   (the weakest change): the measurement says this saves ~nothing.

## Invariants the review must hold (from the R08 GO)

- The candidate cannot suppress, alter, or fabricate its own observation
  (`tests/test_r08_evaluator_isolation.py`).
- Expected values stay controller-side; they never appear in the sandbox script.
- The collector never imports candidate top-level modules.
- Candidate output volume stays bounded.
- A candidate that mutates the tree it is graded on is either detected
  (current: before/after digest) or prevented (proposed: read-only layer) —
  never silently tolerated.
- Grading stays separate from authoritative acceptance mutation (R13).

## Measured context for the decision

- Grading is 0.81 s per tree; with A1, an unchanged tree is graded once per
  experiment run, not once per trial/arm/seed.
- The hidden suite's checks run one command each; the A4 per-attempt saving
  does not apply to them (one check = one command = one sandbox is already
  minimal under the current design).
- Re-scoping option: if the review rules consolidation out of order, A2's
  remaining content (the measurement) is already recorded and the item closes
  as "measured, superseded by A1's caching; no change".

## Acceptance

Acceptance of A2 is the completed adversarial R08 re-review of option 1 or 2
(or a ruling that option 3 / no change is correct), with the review's findings
recorded in `docs/research/` and any accepted variant landing with the R08
tests extended to pin the new isolation boundary.

**Satisfied by the ruling that no change is correct**, recorded in the second
measurement above: the per-check isolation stays as the R08 GO established, no
R08 re-review is commissioned, and the residual cost is booked against
`gate-check-tests` rather than against A2.
