<!-- informative -->

# Independent Review: Bounded Recursive-Planning Pilot — the Model Producer

Review of `853d2a2` ("Never lose a proposal record"), tip of the producer work
(`30b1527` + `853d2a2`), against the mandate in the
[producer review brief](phase4-recursive-planning-pilot-producer-review-brief.md).
The guard was not re-litigated except where the producer uses it. The reviewer is
not the author of these commits, `planning_pilot.py`'s producer, or
`tests/test_planning_pilot.py`.

## Verdict

**NO-GO for wiring the producer into a run.**

Two independent, reproducible falsifications of the mandate's central claim —
a model may propose but may not decide:

1. **(P0) The untrusted provider is handed the live `RunState` and runs before the
   guard derives every bound from that same state.** A provider can rewrite its own
   depth, budget, and trial-count constraints in place, so `admit_proposal` admits
   a structure the pilot forbids. This defeats the derivation the guard's own three
   reviews were built around, and it falsifies "the provider is never trusted".
2. **(P1) The P8 claim is false on common provider inputs.** `except PilotRefusal`
   catches only the three wrapped checks and the coverage-shape check. Provider
   faults, malformed payload assembly, and `admit_proposal`'s own plain
   `DecompositionError` pre-checks all escape with **no record**. `853d2a2` closed
   one instance of the class; it did not close the class.

No production code calls `planning_pilot.propose_decomposition`, so real-world
impact today is nil; both defects are blockers for the wiring the verdict rules on.

## Environment and baseline

- Repo `/home/ruben/Projectos/meta-project/orchestrator`; reviewed revision
  `853d2a2a48a54f69d4d36c51ddb25b502a4de8f0` in a detached worktree at
  `/tmp/kilo/pilot-producer` (`git worktree add --detach … 853d2a2`).
- `./scripts/check-tests --fast` → **829 passed, 30 deselected** (matches brief).
- `./scripts/check-docs` → **251 links OK**; `./scripts/check-static` → gated rules
  clean (advisory B/SIM reported, `--exit-zero`).
- `.venv/bin/python -m pytest -q` → **855 passed, 4 failed** in this environment.
  The four failures (`test_phase3_project_oracle.py` ×3, `test_probe_oracle.py` ×1)
  are sandbox-infrastructure errors (`ORACLE_INFRASTRUCTURE_ERROR`) in files this
  change does not touch; the same test passes in isolation (167 s). Reported as an
  environmental residual, not a producer regression.
- No source edits; probes live under `/tmp/kilo/`; the A5 scratch copy was
  discarding after the experiment. Host worktree clean apart from the untracked
  `.venv` symlink used to run the gates.

## Ruling on concern 1 — the kernel-bound test is not adequate

**Ruling: unacceptable as written; the substring test does not bind the claim and
must be replaced by a parse-and-compare pin.**

Concern 1 is confirmed with a concrete reproducer. On a scratch copy of the tree:

```
cp -a /tmp/kilo/pilot-producer /tmp/kilo/pilot-scratch
sed -i 's/^MAX_PROPOSED_CHILDREN_PER_TRIAL = 32/MAX_PROPOSED_CHILDREN_PER_TRIAL = 99/;
        s/^MAX_PROPOSAL_DEPTH = 2/MAX_PROPOSAL_DEPTH = 7/' \
    /tmp/kilo/pilot-scratch/src/orchestrator/planning_pilot.py
# also set docs/tables/limits.yaml maximum_children_per_node: 99
/tmp/kilo/pilot-scratch/.venv/bin/python -m pytest -q tests/test_planning_pilot.py
# -> 74 passed
```

All 74 pilot tests pass with the enforced depth at 7, the per-trial ceiling at 99,
and the fan-out table at 99, while `PROJECT_KERNEL` still tells the frozen target
"depth 2" and "32-proposed-children-per-trial". The guard's own boundary tests use
`pilot.MAX_PROPOSAL_DEPTH` symbolically, so they cannot catch the drift either.
Concern 1 is a real weakness, not a stylistic one.

The honest fix does **not** interpolate module constants into the kernel string
(that would couple a frozen target's contract text to code state at import time).
Keep the kernel text a literal, and make the test parse the literals and compare
them to the running code:

```python
def test_the_kernel_named_bounds_equal_the_enforced_bounds():
    import re
    from orchestrator import planning_pilot as pilot
    from orchestrator.experiment_target import PROJECT_KERNEL

    text = " ".join(PROJECT_KERNEL["current_non_goals"])
    depth = int(re.search(r"depth (\d+)", text).group(1))
    ceiling = int(re.search(r"(\d+)-proposed-children-per-trial", text).group(1))
    assert depth == pilot.MAX_PROPOSAL_DEPTH
    assert ceiling == pilot.MAX_PROPOSED_CHILDREN_PER_TRIAL
    # The fan-out name must also be numerically bound:
    fanout = int(re.search(r"maximum_children_per_node(?: of)? (\d+)", text).group(1))
    assert fanout == pilot.maximum_children_per_node()
```

This requires adding the fan-out *value* to the kernel text once at admission (the
text currently names `maximum_children_per_node` without a number, so the fan-out
bound is unbound even by parsing). The kernel text stays a frozen literal; the test
becomes a real pin that fails when code drifts.

Secondary: the P6 pin (`"DECLARED" not in repr(payload)`) is likewise a substring
proxy, but unlike P7 the property it guards is structural (no consumer exists), so
it is a weak test of a property that happens to hold. Not gated on it.

## Ruling on concern 2 — inert is acceptable only behind a gate, not as a resting state

**Ruling: "reviewable but unwired" is acceptable for *this* review stage, because
the seam is reviewable without a model; it is not acceptable to leave the producer
unwired once the defects above are fixed, and it is not acceptable as a shipping
state for a safety-relevant path.**

The producer is genuinely unreachable. Search across `src/`, `scripts/`, `tests/`,
and non-`.py` files finds **no** import of `planning_pilot` and **no** call to
`planning_pilot.propose_decomposition`; every `propose_decomposition` call site
(`controller.py:694`, `conformance.py`, `trial_driver.py`, several tests) is the
unrelated `Controller.propose_decomposition`, which builds a payload from
caller-supplied `criterion_coverage` and never calls `admit_proposal`.

That matters for the seam's soundness: `Controller.propose_decomposition` is a
parallel public constructor that admits a decomposition **without the pilot guard**.
As long as nothing routes model output, there is no bypass. The moment a model
adapter is wired, choosing that entry point bypasses the entire pilot. The minimum
integration that makes the producer non-inert without a model:

1. Add a deterministic **scripted provider** arm to the pilot (a provider that maps
   the fixture's parent to a fixed child list), and route it through
   `planning_pilot.propose_decomposition` → `DECOMPOSITION_ADMITTED` on replay, so
   the seam and the record are exercised end to end.
2. Make the model-output path structurally unable to use the unguarded constructor
   (e.g. the pilot entry point is the only one that accepts a provider, and the
   controller path refuses a payload whose source is a model proposal), and add a
   test that the guard runs on that path.
3. Only after findings F1 and F2 are fixed, since both are load-bearing for the
   scripted arm to be meaningful.

## Findings

### F1 (P0) — the provider can rewrite the state the guard derives its bounds from

**Guarantee violated:** the mandate's "it may not decide that structure"; P3
"derivation is mandatory"; and the docstring claim "**The provider is never
trusted**". (The literal wording of guarantee 8 is "through the result"; the vector
here is through the *argument*, which is the stronger defect.)

`propose_decomposition` calls `provider(state, parent_node_id)` and only then calls
`admit_proposal`, which derives depth (`Controller.node_depth`),
`proposed_children_in_trial` (CHILD_OF relations), parent scope (node contract), and
parent budget (`state.ledger.budgets`) from that same live object. A provider that
mutates the object it was handed changes those derived inputs.

**Minimal reproducer** (full probe `/tmp/kilo/probe_provider_state.py`):

```python
from orchestrator import planning_pilot as pilot
st = _state(GOOD, chain_depth=pilot.MAX_PROPOSAL_DEPTH - 1)  # p at depth 2: must refuse

def provider(s, p):
    s.relations.clear()                                     # p now reads depth 0
    s.ledger.budgets["p-budget"].limits["MODEL_COST"] = 1e12  # raise own limit
    return [_child("c1", "src/good/x.py", 1e9)]              # 1e9 > real limit 10

payload, record = pilot.propose_decomposition(st, "p", provider, proposal_id="P")
```

Observed: `payload is not None`; `record == {'admitted': True, 'refusing_stage':
None, 'proposed_children': 1, 'accepted_children': 1, 'depth_reached': 0}`;
`st.ledger.budgets["p-budget"].limits == {'MODEL_COST': 1e12}`.

Expected: refused at `BOUNDS` (parent at depth 2) and at `BUDGET` (1e9 > 10), with
the provider's objects and the run state untouched.

A second probe shows a refused proposal still leaves the provider's mutation in
state: a provider that inserts `state.nodes["ghost"]` before returning a
scope-escaping child yields `payload=None`, `refusing_stage='SCOPE'`, and
`"ghost" in st.nodes` is `True`. Refusal is total over the *payload* but not over
the provider's side effects.

**Proposed permanent regression test:** pass a mutating provider and assert (a)
the state is unchanged (compare a canonical serialization/hash before and after),
and (b) the outcome is identical to a non-mutating provider proposing the same
children. **Design fix:** derive all bounds from the authoritative state *before*
invoking the provider, and hand the provider a read-only view or an isolated deep
copy — never the live `RunState`. This requires a P7a spec amendment, since P7a
currently specifies the provider as `(state, parent_node_id)`.

### F2 (P1) — P8's "every proposal produces a record" is false; the 853d2a2 fix closed one instance, not the class

**Guarantee violated:** guarantee 6 and P8 ("Every proposal produces a record
naming its outcome and the refusing stage"). The mandate calls the record
"the structural propert[y] that matter[s]".

`853d2a2` moved coverage derivation inside the `try` and changed its shape refusal
to `PilotRefusal("COVERAGE")`. That closes exactly the malformed-`criterion_ids`
instance. It does **not** close the class, because the net is still
`except PilotRefusal` and throws its net only around that scope.

**Minimal reproducer** (full probe `/tmp/kilo/probe_a3.py`):

```python
payload, record = pilot.propose_decomposition(
    _state(GOOD), "p", lambda s, p: [], proposal_id="P")
# raises DecompositionError("a proposal must declare at least one child")
# -> no record, no payload
```

Observed: `RAISED DecompositionError: a proposal must declare at least one child`.
Expected: `(None, {'admitted': False, 'refusing_stage': <stage>, ...})` — a provider
returning `None`/`[]` is a refusal that must be instrumented, exactly as the
malformed-coverage refusal now is.

**Proposed permanent regression test:** for each input in the A3 list below, call
`propose_decomposition` and assert it returns a `(payload, record)` tuple (never
raises) with exactly one record whose `admitted` matches. **Fix:** widen the guard
to `try: … except PilotRefusal as e: stage=e.stage; except DecompositionError as e:
stage="GUARD"; except Exception as e: stage="PROVIDER"` around the provider call,
normalization, admission, coverage derivation, and payload assembly. Stage names
must distinguish a guard refusal from a provider fault so the record stays honest.

### F3 (P2) — a bare `DecompositionError` refusal can escape the stage wrap entirely

**Guarantee violated:** guarantee 7 ("the refusing stage is the stage that fired")
for the pre-check refusals. `admit_proposal` wraps only `check_bounds`,
`check_scope_containment`, and `check_budget_authority`. Its own earlier refusals
are plain `DecompositionError`:

- empty proposal (`propose_decomposition` reachable via `[]`/`None`);
- parent unknown / declares no owned scope;
- parent budget not registered;
- `state`/`parent_node_id` `None`.

Observed in `/tmp/kilo/probe_a4.py`: `empty proposal -> RAISED DecompositionError
(NO RECORD)`. Expected: a record with a stage (`BOUNDS`/`GUARD`), not an
uninstrumented bare exception. This is the same defect as F2 seen from A4; F2's fix
covers it. Included separately because A4 asks whether a refusal can escape
unwrapped, and it can.

### F4 (P2) — the returned payload is mutable and scope containment is not re-checked by the reducer

**Guarantee implicated:** guarantee 1 ("the seam cannot be bypassed"). This is
F1-adjacent but distinct: it needs no provider misbehaviour, only a caller that
mutates the payload after `propose_decomposition` returns.

```python
ppayload, _ = pilot.propose_decomposition(
    st, "p", lambda s, p: [_child("c1", "src/good/x.py")], proposal_id="P")
ppayload["children"][0]["contract"]["owned_scope"][0] = "src/evil/x.py"
validate_decomposition(ppayload, st)   # -> returns None: accepted
```

Observed: the mutation survives the reducer's re-validation, because
`validate_decomposition` does not enforce scope containment (the F1 single-layer
residual, explicitly out of scope for the guard). The producer's deep copy protects
the provider's objects and the provider→admitted direction, but the caller holds a
mutable reference to the admitted payload. Also demonstrated: a hand-built payload
with an escaping child scope is accepted by `validate_decomposition` with
`admit_proposal` never running (`/tmp/kilo/probe_a1_a6_a7.py`).

This is only a bypass in the sense that the *only* enforcement of scope containment
is the pilot guard (F1, out of scope). Reported at P2 as a wiring constraint: any
integration must ensure the payload is not mutated between the guard and the event
admission, or the reducer must gain the scope check (a change to the harness
contract that F1 forbids). Not counted against the producer's own logic.

### F5 (P2) — `depth_reached` records the parent's depth, not the depth reached

**Guarantee implicated:** guarantee 6 ("the record's counts agree with the
outcome") / P8 "the depth reached". For an admitted proposal from a parent at
depth 1, `depth_reached` is `1` (the parent's depth), not `2` (the depth of the
newly admitted children). `/tmp/kilo/probe_a3.py` control: parent `p` at depth 1 →
`depth_reached: 1`. The name and the spec's wording do not match the value. Low
impact; pin the intended semantics or rename the field.

## Guarantees under review — attempt and outcome

1. **Seam cannot be bypassed** — **FALSIFIED (F1).** A provider can rewrite the
   state the guard derives depth/scope/budget/count from, producing an admitted
   payload the guard's derivation would otherwise refuse. Attempts that held:
   provider mutates its returned list after the fact (deep copy isolates it);
   generator provider (consumed, then deep-copied); child with surprising
   `owned_scope` / `budget_allocation` types (refused, wrapped); reusing a child
   admitted under another parent (re-derived and re-checked); post-return payload
   mutation (not caught by the reducer, F4).
2. **Refusal is total through the producer** — **attacked, not falsified.**
   `[valid, invalid]` in both orders and `[valid, bad, valid2]` all return
   `payload=None`, `accepted_children=0`, correct stage, and leave the provider's
   objects byte-identical (`/tmp/kilo/probe_a2_a8.py`). Caveat: F1 shows the
   *state* is not protected, only the payload and provider objects.
3. **Coverage is derived, not supplied** — **attacked, not falsified.** A child
   omitting `criterion_ids` yields empty coverage and the reducer refuses
   (`mandatory parent criteria not covered`); a top-level `criterion_coverage` key
   smuggled into a child is ignored; an optional child credited with a mandatory
   criterion is refused by the reducer. Division of labour is real.
4. **P6: no `DECLARED` target** — **attacked, not falsified.** The payload contains
   none of `obligations_source` / `non_derivable_axes` / `non_derivable_reason` and
   is never fed to `build_target`, which takes a target description, not a
   decomposition payload. No consumer of the payload exists at all.
5. **P7: kernel bounds are the enforced bounds** — **FALSIFIED (concern 1).**
   Scratch-copy experiment changes the enforced depth/ceiling/fan-out without
   breaking the kernel test. Ruling above.
6. **Exactly one record per proposal** — **FALSIFIED (F2).** Thirteen no-record
   input classes; see A3.
7. **Refusing stage is the stage that fired** — **partially falsified (F3).** The
   four wrapped stages are correctly labelled; pre-check refusals escape unwrapped.
8. **Provider cannot mutate state through the result** — **falsified in spirit
   (F1).** Through the result it holds (deep copy); through the provider argument it
   fails completely, which is the vector that matters for an untrusted model
   adapter.

## A3 — complete list of no-record paths

Every path below returns nothing or raises without producing a record, verified in
`/tmp/kilo/probe_a3.py`. Grouped by where the exception escapes:

**Outside the guarded `try` (provider call + normalization, `planning_pilot.py:452-453`):**

1. provider is not callable, or its call raises any exception;
2. provider returns a non-iterable (`int`, coroutine) → `TypeError`;
3. provider returns an iterable containing a non-mapping element (`int`, `str`) →
   `TypeError` / `ValueError`;
4. provider returns a generator that raises during iteration → provider's exception;
5. provider returns a `Mapping` (iterates keys) → `dict(str)` `ValueError`.

**Inside `admit_proposal` but a plain `DecompositionError`, so `except PilotRefusal` misses it:**

6. empty children (provider returns `[]`, `None`, or an empty mapping);
7. parent node unknown, or parent declares no `owned_scope`;
8. parent budget not registered;
9. `state is None` or `parent_node_id is None`.
   (The `parent_depth` / `already_proposed` / `parent_scope` assertion refusals are
   not reachable from `propose_decomposition`, which does not pass those kwargs.)

**Inside a wrapped check but a non-`DecompositionError` escapes the wrap:**

10. child `contract` is not a `Mapping` (`int`/`str`) → `AttributeError` in
    `check_scope_containment`. Any other unexpected non-`DecompositionError` behaves
    the same.

**Inside `_criterion_coverage` but not a `PilotRefusal`:**

11. child has `criterion_ids` but no `node_id` → `KeyError: 'node_id'`
    (`planning_pilot.py:417`).

**Payload assembly outside the `try` (`planning_pilot.py:471-482`):**

12. `dependencies` entry is not a `Mapping` → `TypeError`;
13. `expected_versions` entry is not a `Mapping` → `TypeError`.

## A5 — kernel-drift experiment (scratch copy, since discarded)

Commands and result are in the ruling on concern 1. Summary: with
`MAX_PROPOSAL_DEPTH = 7`, `MAX_PROPOSED_CHILDREN_PER_TRIAL = 99`, and
`maximum_children_per_node: 99`, `tests/test_planning_pilot.py` reports **74
passed** and the specific kernel test reports **1 passed** against a kernel string
that still claims "depth 2" and "32-proposed-children-per-trial". Scratch copy
removed (`rm -rf /tmp/kilo/pilot-scratch`).

## Claims verified by pre-image reading

Pre-images read via `git show 431e87f:…` and `git show 30b1527:…`. Semantic deltas:

- `30b1527` adds `PilotRefusal`, wraps the three guard checks (message preserved,
  type narrowed to a `DecompositionError` subclass so existing contracts hold),
  adds `_criterion_coverage` + `propose_decomposition`, amends the kernel non-goal,
  and adds tests. No existing guard semantics changed.
- `853d2a2` moves coverage derivation inside the `try` and changes its shape
  refusal from `DecompositionError` to `PilotRefusal("COVERAGE")`, plus two tests.
  The change is exactly one instance of the F2 class; the class remains open
  because the provider call/normalization and `admit_proposal`'s pre-checks are
  still outside the `except PilotRefusal` net.

The producer's `raw = provider(...)` / `children = [...]` lines are byte-identical
in `30b1527` and `853d2a2`, confirming the fix did not touch the provider boundary.

## Residual unknowns

- **No model adapter** (disclosed concern 3). The provider contract — sync/async,
  error shape, mapping normalization — is unspecified, so "a real adapter would…"
  is untestable here. The producer's shallow normalization (`dict(child)`) rejects
  a single-mapping provider and a `Mapping` return; whether that is intended is
  unstated.
- **Integration is unproven** (disclosed concern 2). No end-to-end run exercises
  the producer, so F1's persistence into a live run's state is inferred from the
  object identity of `RunState`, not observed through a controller.
- **`_nesting_shape` one-level residual (F10)** and the unwired instruments are out
  of scope and were not tested.
- The four full-suite sandbox failures are environmental; they pass in isolation and
  touch no changed file. Not reproduced against a pre-change revision.
- The reducer's sufficiency check was probed whole-path for the omission and
  optional-cover cases only; a full adversarial campaign against
  `validate_decomposition` is the guard's (cleared) surface, not this review's.

## Reproduce everything

All probes under `/tmp/kilo/`, run from the worktree with the imported source:

```
git worktree add --detach /tmp/kilo/pilot-producer 853d2a2
ln -s /home/ruben/Projectos/meta-project/orchestrator/.venv /tmp/kilo/pilot-producer/.venv
cd /tmp/kilo/pilot-producer
PYTHONPATH=src /home/ruben/Projectos/meta-project/orchestrator/.venv/bin/python /tmp/kilo/probe_a3.py
PYTHONPATH=src /home/ruben/Projectos/meta-project/orchestrator/.venv/bin/python /tmp/kilo/probe_a2_a8.py
PYTHONPATH=src /home/ruben/Projectos/meta-project/orchestrator/.venv/bin/python /tmp/kilo/probe_a4.py
PYTHONPATH=src /home/ruben/Projectos/meta-project/orchestrator/.venv/bin/python /tmp/kilo/probe_a1_a6_a7.py
PYTHONPATH=src /home/ruben/Projectos/meta-project/orchestrator/.venv/bin/python /tmp/kilo/probe_provider_state.py
./scripts/check-tests --fast
./scripts/check-docs
./scripts/check-static
```

Probe inventory:

- `probe_a3.py` — builder `state(chain_depth, with_scope, with_budget)` and
  `child(node_id, scope, amount, criterion, dimension)`; drives the 13 no-record
  inputs and the admitted control.
- `probe_a2_a8.py` — partial-acceptance orders, provider-object integrity, payload
  mutation and re-admission, deep-copy checks.
- `probe_a4.py` — stage labelling for depth/fan-out/scope/budget/coverage and the
  unwrapped empty-proposal refusal.
- `probe_a1_a6_a7.py` — hand-built payload accepted by `validate_decomposition`
  without the guard; post-return scope mutation surviving reducer re-validation;
  coverage injection attempts; payload key scan.
- `probe_provider_state.py` — the F1 reproducer (ledger inflation, relation
  clearing, refused-path ghost node).

## What this review did not cover

- The guard's own bounds (cleared at re-review 3): depth derivation, per-node
  fan-out, trial ceiling, scope containment, budget authority, deep copy — attacked
  only where the producer uses them (F1).
- The F1 single-layer containment decision, the `_nesting_shape` residual, the
  model adapter, and unwired instruments.
- A full adversarial campaign against `validate_decomposition`; this review tested
  only the producer's payload through it.
