<!-- normative for harness; the measurement procedure is informative -->

# Phase 3 design brief — project-owned oracle (self-host)

Status: **design brief**. This document specifies the acceptance manifest whose
visible and hidden checks are a real repository's own gates, with the
meta-registry floors frozen as gates and bound R12-style — the self-host
roadmap's [Phase 3](../self-host.md#phase-3-project-owned-oracle). The
generator/oracle independence discipline and the R08 probe-oracle ruling
(`docs/research/phase2-remediation-re-review.md`) govern every decision here.

## Scope and the seam

The fixed target's oracle (`build_hidden_oracle`) hardcodes the synthetic
four-component hidden suite. Phase 3 replaces that construction for a real
repository: an **acceptance manifest builder** (`build_project_oracle`) whose
checks are the target's own gates, plus the **meta-registry floors** frozen as
checks in their own right. The seam is `oracle.OracleManifest` itself: the
builder produces ordinary checks in the sanctioned `sandboxed_probe` shape
(data-only driver, controller-side compare), so the whole grading stack
(`_run_probe`'s per-check copy, tamper digest, harness deadline; `freeze_manifest`
's probe-shape enforcement; `probe_identity` binding) applies unchanged.

Nothing about candidate production changes: this phase is the oracle side. The
patch path (Axis 1) and the general target builder (Phase 2) already run.

## The manifest shape

The visible/hidden split is the existing architecture, not a new one:

- **Visible checks** are the attempt's verification commands
  (`target.visible_commands`, run per attempt by `verify_commands_for`); their
  results are the acceptance evidence the model's work is judged on during the
  run.
- **Hidden checks** are the oracle manifest's checks (grading only). Their
  identity, expectations, and results never enter the prompt, the tool
  results, or the sandbox script (R08); the split between visible and hidden
  gates is oracle-side metadata.

For the orchestrator repository the in-manifest checks are:

| check | kind | criterion |
|---|---|---|
| `gate-check-tests` | command gate (`scripts/check-tests --fast` over the frozen deterministic corpus) | `AC-TESTS` |
| `gate-check-static` | command gate (`scripts/check-static`) | `AC-STATIC` |
| `registry-counterexample-floors` | frozen floors (`min_covered`, `prefix_revision` completeness, mapped-pin rules) | `AC-REGISTRIES` |
| `registry-open-defect-floors` | frozen floors (`min_fixed`, `min_open`, pin-structure rules) | `AC-REGISTRIES` |
| `registry-guard-floors` | frozen floor (`min_parameters`) | `AC-REGISTRIES` |
| `registry-fidelity-floors` | frozen ratchet (registered-fake rows + declarations) | `AC-REGISTRIES` |

`orchestrator.conformance` is a **trusted-side gate**, not a grading check
(measured constraint below): `build_project_oracle` runs
`conformance.run_full_suite()` and refuses to build a manifest unless it
passes — the same fail-closed posture as the `NAMESPACE_V1` capability probe,
and `run_experiment` already gates on it before pilots.
`scripts/check-docs` is also trusted-side because its external-link check
needs the network denied by the grading sandbox's net namespace. The manifest
therefore has three criteria (`AC-TESTS`, `AC-STATIC`, `AC-REGISTRIES`), not
five.

**The measured constraint (nested execution).** The grading sandbox blocks
nested sandbox construction: a launcher inside the workload cannot write
`/proc/<pid>/setgroups` (EACCES), so `SandboxRunner` reports
`NAMESPACE_V1` unavailable from within any grading sandbox. Two consequences,
both measured on this host:

1. `orchestrator.conformance` *is* sandbox self-testing (`_oracle_checks` runs
   the hidden suite in sandboxes) — it cannot grade inside a grading sandbox,
   hence trusted-side.
2. The repository's test suite contains sandbox-dependent modules whose
   fail-closed posture turns the nested unavailability into test failures.
   `gate-check-tests` therefore runs the suite's **deterministic subset**: the
   frozen argv is `scripts/check-tests --fast -p no:cacheprovider` plus one
   `--ignore` per runner-spawning module (17 at freeze time), explicit paths
   for the frozen baseline test modules, and two named exclusions for nested-
   environment-sensitive tests. Gate-control files (ruff/pytest config,
    conftests, startup hooks) and the baseline `tests/**/*.py` corpus are
    frozen by exact path set and content hash; any test/config edit, addition,
    or deletion requires deliberate re-freeze (the frozen executable test
    corpus is immutable within a run). At build, every frozen gate/config/test/
    registry blob is also checked against `TargetSnapshot.commit`; a dirty
    worktree cannot silently supply policy for a different target snapshot.
    The exclusions are frozen in the argv,
   baseline hashes are held controller-side, and `-I`/`PYTHONSAFEPATH`
   prevent candidate `sitecustomize.py` from running before those checks. A
   *new* runner-spawning
   module makes the gate go red on a good tree — a fail-closed drift that is
   visible, never silent. The excluded modules are the trusted gate's
   (the full `scripts/check-tests` and `conformance.run_full_suite()` run
   host-side at build and at run start).

**The frozen gate identity.** The in-manifest command gates
(`check-static`, `check-tests`) live in the candidate's tree — a candidate
could rewrite one to `exit 0`. Each therefore freezes the **content hash of
its gate script** in the compare closure and verifies the collected script
bytes before accepting the exit code; the gate-control configuration and
baseline test corpus are likewise frozen by path/hash. Editing any requires a
deliberate re-freeze (mismatch rejection, the registries' own philosophy
applied to the gates themselves). `check-docs` is trusted-side and runs on the
host as a build precondition; it is **not** graded on candidate edits, since
its external-link half cannot run in the sandbox's network namespace. That
trusted-side scope limit is explicit, not an implicit claim of candidate
coverage.

### The gate execution model

A command gate runs the **candidate's tree**, so
the frozen suite actually grades the candidate's source; the oracle-definition
surface is shadowed by read-only overlays of the trusted bytes:

- Every frozen gate script, gate-config file and collected test file is bound
  read-only over the candidate's own copy. A candidate therefore cannot make the
  gate it is graded by lie, and the overlay is defence in depth behind the
  identity check rather than the only defence.
- `conftest.py` is deliberately **not** overlaid. pytest executes it as
  candidate code, so shadowing it would mask a candidate that hangs or
  deselects its own run: a modified `conftest.py` is rejected by the frozen
  identity check, and an unrunnable one surfaces as `CANDIDATE_TIMEOUT` instead
  of being silently hidden behind the frozen copy. This is what makes the
  `known-timeout` row of the corpus below reachable.
- The frozen stage is materialised in a cache directory **outside `/tmp`**. The
  sandbox mounts a private tmpfs over `/tmp` before it binds the oracle's
  overlays, so a stage under `/tmp` is invisible at mount time and every
  overlay fails with `EPERM`. It is content-addressed by the digests it holds,
  so rebuilding a manifest for the same target reuses one directory.
- Candidate **source is graded, never frozen**. Only `tests/**` (the collected
  corpus) and `scripts/**`/gate config are frozen; `src/**` is neither
  identity-pinned nor overlaid. Freezing it would make the acceptance gate
  vacuous — only a tree byte-identical to the freeze could pass.
- A candidate-added file carrying a frozen **control basename** anywhere in the
  tree is rejected, not just at frozen control paths: a nested `ruff.toml` or
  `pytest.ini` cannot appear in a freeze-time inventory, but it can shadow the
  gate's own configuration.
- The per-check **execution tree is scratch**: the sandbox bind-mounts it
  read-write, so a gate that runs the target's own toolchain leaves
  `__pycache__` and `.pytest_cache` behind. The tamper guarantee is on the
  **candidate copy**, which must be byte-identical after the probe; the
  execution tree is exempt for probes that declare they write.
- Identity and corpus membership are verified **after** the probe returns. They
  exist to catch an oracle-definition rewrite, not to replace the gate, so
  running the gate first reports a hanging candidate as the timeout it is rather
  than as an identity mismatch on a file the trusted side already knows differs
  from its freeze.
- The collected corpus is a **closed set**. Adding, removing or renaming a
  collected module is rejected by directory-membership comparison, because
  every frozen file can still hash correctly while the corpus is no longer the
  one the trusted side froze.

Bidirectional coverage (the freeze's `validate_bidirectional_coverage`) holds
between these three criteria and the target's obligations — the exit gate's
"bidirectional coverage validation" is this check plus the corpus below.

## The hidden checks' construction

**Command gates** are `sandboxed_probe`s whose driver runs the gate argv in
the per-check sandbox and reports `COLLECT:{"exit": …, "gate_hash": …}` plus
path/hash inventories for the frozen gate-control files and baseline test
corpus; the compare (controller-side) checks every identity against its
freeze-time hash before considering `exit == 0`. Decisions:

- The gate argv is **exact and frozen** in the check (bound by `probe_identity`).
  `-p no:cacheprovider` keeps pytest's `.pytest_cache` writes out of the
  graded tree: the tamper digest (R08) must not fire on a gate's own
  bookkeeping, and an unweakened digest is worth the flag. `check-static`
  likewise runs with its cache disabled; `check-docs` writes nothing in-tree.
- The **environment** is the sandbox's (`_HIDDEN_ENV`: `HOME=/tmp`,
  `PYTHONDONTWRITEBYTECODE=1`, a fixed `PATH`), extended by one read-only
  mount: the project's `.venv` at `/work/.venv` — the gates' interpreter
  (pytest, ruff) is toolchain, and the toolchain view is declared in the
  frozen environment identity. The gate scripts resolve
  `$PROJECT_ROOT/.venv/bin/python` exactly as on the host.

**Registry-floor checks** are `sandboxed_probe`s whose driver parses the
candidate's registry/test bytes and reports **metrics and identities** as the
`COLLECT` payload (facts only); the compare — controller-side — evaluates
those facts against the **frozen floors and identity sets** captured at
manifest-build time (`min_covered ≥ 23`, `prefix_revision` present for every
finding, every freeze-time covered finding and mapped node id still present,
`min_fixed ≥ 10`, `min_open ≥ 0`, every freeze-time fixed pin and guard
parameter identity still present, `min_parameters ≥ 17`, every freeze-time
fake identity/deviation still present, the fidelity ratchet's row count ≥ its
frozen count). This is the point of the phase: a candidate that lowers a
floor in the YAML, demotes a covered finding, or deletes a declared deviation
**trips the frozen gate** even though the repository's own meta-tests would
pass the edited registry. "A candidate that adds a fixture or closes a
declared deviation can trip a floor" — those are graded against the frozen
floors and the structural rules, so the registries are part of the
deliverable: a legitimate fixture addition or deviation closure must move the
floor deliberately (visible in review), exactly as the registries' own
comments demand.

## The R12 binding: mismatch rejection, not a label echo

The frozen floors travel **inside the compare closures**, and `probe_identity`
binds a probe's source plus its closure values (re-review N4/N5). Therefore:

- `oracle_target_binding(target, manifest)` and `hidden_suite_manifest_hash`
  change when any frozen floor, gate argv, or expectation changes — the
  binding *describes* the gates rather than echoing a label.
- `run_trial`, `calibrate_oracle`, and `freeze_manifest` recompute the binding
  and refuse a mismatch; `freeze_manifest` additionally refuses any check that
  is not the sanctioned `SANDBOXED_DATA_DRIVER` shape (re-review N5). A
  hand-built manifest whose floors differ from its own probes is refused.

## Generator / oracle independence

The task statement and obligations never derive from the check construction;
the hidden checks' argv, floors, and expectations never appear in the prompt
or any model-visible context (`filter_context` already drops hidden-pattern
references; nothing oracle-side is put in `input_references`). The corpus
below classifies *trees*, not model outputs — the classifier and the check
construction are what the gate exercises, so no candidate generator is
involved.

## The five-case corpus on a real repository (the exit gate)

The corpus runs over the orchestrator repository itself (a real, non-synthetic
target), through `calibrate_oracle` with `CalibrationCases` and the frozen
manifest, expecting exactly `_CALIBRATION_CASES`' classifications:

| case | tree | controller | expected |
|---|---|---|---|
| `known-good` | the intact repository plus a benign source edit (registries at their real floors) — the accepted tree is not a byte-identical replay of the freeze | `ACCEPTANCE_CLAIMED` | `CORRECT_ACCEPTANCE` |
| `unchanged` | the base: one covered finding demoted to pending and `min_covered` lowered to match (internally consistent — only the *frozen* floor catches it) | none | `CORRECTLY_UNRESOLVED` |
| `known-broken` | the base plus a wrong fix: a mapped regression test deleted (now the repository's own meta-gate fails too) | none | `CORRECTLY_UNRESOLVED` |
| `known-timeout` | the good tree plus a `conftest.py` that sleeps forever at import (the candidate hangs its own gate, so `conftest.py` is executed rather than overlaid) | `ACCEPTANCE_CLAIMED` | `FALSE_SUCCESS` |
| `infrastructure` | the good tree with an injected oracle infrastructure failure | none | `INDETERMINATE` |

The base's defect is chosen so the *frozen floors* are the decisive gate (the
edited registry is internally consistent) — the phase's whole claim. The
timeout case is a genuine candidate-hangs-gate case (never an injected
timeout), and the expected timeout lands on `gate-check-tests` exactly like
the synthetic corpus's `expect_candidate_timeout`.

Cost: every case runs the full hidden gate set over a per-check disposable
copy; `gate-check-tests` is the dominant term (a nested `--fast` suite run).
The corpus is `slow`-marked with an explicit generous timeout — the
pre-commit gate (`check-tests --fast`) deselects slow tests, so the exit gate
costs the development loop nothing, and the plain suite runs it in full.

## Exit gate (informative)

Bidirectional coverage validation plus the five-case corpus classify all five
candidates correctly on the real repository under the frozen floors — and the
`unchanged` case fails *only* because the frozen floor disagrees with the
candidate's edited registry, which the phase exists to prove.
