<!-- informative -->

# Post-experiment decisions

The seventh milestone resolves the [Deferred Design Questions](roadmap.md#deferred-design-questions) on the evidence
the scripted experiment provides, and records the retention and data-classification policy of the observability
backend. This document summarises the record; the machine-checkable record is
`src/orchestrator/decisions.py`, and the demonstrations are the eight tests in
`tests/test_milestone7_acceptance.py`, named after the roadmap bullets.

## The measured comparison, interpreted

`decisions.interpret_comparison` derives the per-arm structural findings from a recorded
[ComparisonReport](experiment.md#evaluation) — every statement is computed from the recorded trials and arm summaries,
so the record cannot drift from the experiment output it interprets:

- DIRECT: the root executes undecomposed; no decomposition is admitted.
- FIXED_ONE_LEVEL: the root decomposes into a fixed one-level tree.
- RECURSIVE_LAZY: decomposition recurs while children remain uncertain; attempts run sequentially.
- RECURSIVE_CONCURRENT: the same recursion, with independent actions scheduled concurrently and the reference speedup
  metric recorded against the sequential arm.

Per-arm outcome distributions are identical in kind across arms because every arm ran the same deterministic client
under the same acceptance authority: no scripted trial produced a false success.

## The limit of scripted evidence

The recorded limit (frozen in `decisions.SCRIPTED_EVIDENCE_LIMIT`): a deterministic scripted client differentiates
orchestration structure, not model behaviour — the recorded per-arm differences are attributable to the decomposition,
scheduling, and budget machinery, and no measured number in the comparison estimates how a real reasoning model would
behave in any arm. The manifest itself freezes its `unmeasured_metrics` disclosure
([experiment.md](experiment.md#evaluation)).

## Resolved decisions and their real-model conditions

Each resolution below is recorded with its demonstrated basis in `decisions.DEFERRED_DECISIONS`, together with the
conditions under which a real-model run would change it:

- **Event store** — SQLite in WAL mode is retained as the production event store
  ([control-plane.md#mvp-ordering-concurrency-and-transactions](control-plane.md#mvp-ordering-concurrency-and-transactions)).
  Nothing normative depends on store-specific behaviour: the reducer and replay contract require only insert-once,
  totally ordered events plus a crash-atomic commit, proven against the chosen store by the standing
  crash-injection and replay-equivalence suites. Reopen on demonstrated multi-writer, replica, or volume requirements.
- **Dispatch substrate** — in-process dispatch is retained
  ([control-plane.md#crash-recovery](control-plane.md#crash-recovery)). `ACTION_DISPATCHED` stays durable before any
  executor contact; the substrate holds no normative state; recovery keeps the `ATTEMPT_FAILED_ON_RECOVERY` path.
  Reopen when real attempts must outlive a controller restart, when executors become resumable, or on multi-host
  execution — any adopted substrate must still hold no normative state.
- **Observability backend** — a read-only query layer over the audit output
  (`src/orchestrator/observability.py`) exposes run history, ledger, evidence, and oracle grades beyond the harness
  API, with the retention and data-classification policy recorded in that module. It adds no mutation path. Reopen on
  high-cardinality usage metrics, model-content retention obligations, or external dashboard requirements.
- **Human-approval integration** — approvals are admitted as typed events through the one admission path with
  source-authority, schema, and policy validation; timeout and escalation are recorded events
  ([control-plane.md#human-approval-lifecycle](control-plane.md#human-approval-lifecycle)). There is no out-of-band
  override. Reopen for human-in-the-loop runs with real approval latencies or delegated authorities.
- **Secrets management** — scoped short-lived credentials are minted by a secrets source
  (`src/orchestrator/secrets.py`) at the sandbox and executor boundaries; both boundaries are audited against the
  source; credential redaction at every boundary is preserved; issuance and retirement are audited as events, and only
  redacted digests are recorded ([control-plane.md#credential-lifecycle](control-plane.md#credential-lifecycle)).
  Reopen when a real provider endpoint brings real credential rotation and revocation requirements.
- **Model-provider abstraction** — a provider interface and registry (`src/orchestrator/model_provider.py`) with
  explicit selection by provider id and exactly one bound adapter per controller; a second adapter can be added as an
  implementation without touching the controller, reducer, or experiment. Routing stays deferred: no multi-model
  routing, no learning-based routing, and no recursive planning on model outputs. Reopen when two adapter
  implementations and recorded verified performance by task type exist.

Still excluded, with their recorded conditions: real-model measured runs (requires an explicit owner decision on model
access and spend recorded in the frozen manifest) and multi-model routing (requires two adapter implementations and a
recorded routing policy).

## Retention and data classification

Recorded in `src/orchestrator/observability.py` and enforced on read:

- the authoritative event log is retained in full — it is authoritative state and the replay source, and no observability
  query prunes it;
- every query result is a rebuildable projection of the log, never authoritative;
- hidden-check evidence is retained for grading and audit and reads back classified RESTRICTED, summarised by outcome;
- credential records carry redacted digests only (RESTRICTED); raw credential values are classified SECRET, are never
  persisted, and are never queryable;
- event history, ledger, node, attempt, approval, and model-output projections are INTERNAL.
