RTIN Structural Partitioning and Recursive Resolution

Objective

Determine whether phase-partitioned inference and bounded recursive execution improve observable task completion, evidence integrity, and false-completion calibration under explicit resource budgets.

This contract authorizes no performance claim by itself. The current result state is AWAITING_LOCAL_RUN.

The benchmark separates four questions:

1. Does Beginning-Middle-End partitioning outperform uninterrupted inference?
2. Does typed state outperform raw-text or prose-summary handoff?
3. When does recursion improve quality enough to justify coordination cost?
4. Does source-linked collapse preserve recoverability better than ordinary recursive summarization?

Conditions

C0 — Single pass

One model call receives the source material, task, output contract, and completion tests and must produce the final artifact.

C1 — Same-context plan and answer

One model call plans and executes within one context. No fresh phase conditioning is provided.

C2 — Linear TSP, raw handoff

Three invocations of the same pinned model:

Beginning -> Middle -> End

The original prompt and all prior raw output are concatenated.

C3 — Linear TSP, compact-summary handoff

The Beginning and Middle outputs are compressed into bounded prose summaries before the next phase.

C4 — Linear TSP, typed-state handoff

The same three phases communicate through schema-validated Task Frames and Resolution Packets. Raw outputs remain available by reference.

C5 — Recursive TSP, typed-state handoff

The Middle phase may create one level of bounded child cells. Every child must produce a source-linked Resolution Packet.

C6 — Stronger-model reference

A disclosed stronger model runs the single-pass and best structured condition. This is a reference, not a matched-resource substitute.

Required suites

1. Single-pass and same-context plan-and-answer baselines
2. Linear TSP with raw-text handoff
3. Linear TSP with compact-summary handoff
4. Linear TSP with schema-validated typed-state handoff
5. One-level recursive TSP with reversible Resolution Packets
6. Matched-total-compute comparison
7. Matched-peak-memory comparison
8. Evidence recoverability and false-completion challenge set
9. Cross-model and same-model role ablations

Task families

Repository transformation

A task includes a versioned repository, required multi-file change, stated invariants, deterministic tests, and a patch receipt.

Long-document synthesis

A task distributes evidence, caveats, minority findings, and contradictions across a long source set. The result must retain exact source references.

Structured artifact generation

A task requires a schema-valid artifact with mandatory and prohibited fields.

Dependency-aware planning

A task contains prerequisites, parallelizable work, blockers, and acceptance gates.

Adversarial closure

A task contains an unmet requirement that a fluent model may overlook. The benchmark measures false completion.

Resource views

Every result must be reported under both:

Matched peak memory

Conditions use comparable peak accelerator memory. Additional calls and total tokens remain visible.

Matched total compute

Conditions receive comparable aggregate inference compute or an explicitly disclosed proxy such as total processed tokens and model invocations.

A condition may win one view and lose the other. Both results are retained.

Invalid-run conditions

A run is invalid when any of the following occurs:

• source material or task instructions differ across conditions;
• model, tokenizer, quantization, runtime, prompts, or tool availability differ without disclosure;
• recursion, repair, retrieval, or verifier calls are unbounded;
• fixture conformance is presented as model performance;
• failed child calls or orchestration costs are excluded;
• benchmark tasks were used to tune phase prompts or recursion thresholds without disclosure;
• human raters can see condition labels;
• raw task rows, receipts, or source-recovery traces are missing;
• a condition receives extra evidence or precomputation that is not charged or disclosed.

Primary metrics

Report every metric separately:

• requirement_coverage
• completion_test_pass_rate
• source_fidelity
• unsupported_claim_rate
• contradiction_rate
• false_completion_rate
• artifact_validity
• evidence_recoverability
• open_loop_preservation
• cross_child_consistency

Resource metrics

• input_tokens_total
• output_tokens_total
• model_invocations
• peak_vram_bytes
• wall_time_ms
• orchestration_time_ms
• network_bytes
• energy_joules
• estimated_cost_usd

Topology metrics

• recursive_depth
• child_cells
• split_precision
• split_recall
• merge_conflicts
• evidence_escalations
• cache_hit_rate

Evidence recoverability protocol

For every material root claim:

1. record the root claim identifier;
2. follow its child-resolution references;
3. identify the exact source span or artifact;
4. verify that the source supports the claim;
5. record the number of escalations and bytes retrieved;
6. fail the row when the evidence path is broken or points only to an unsupported summary.

False-completion protocol

Each task includes machine-readable completion tests. A condition commits success only when the tests pass or when the result explicitly identifies the unmet condition.

A false completion occurs when the system declares or implies completion while one or more required tests remain unsatisfied.

Recursion trigger evaluation

The control plane records:

• estimated recursion pressure;
• decomposability score;
• expected focus gain;
• estimated coordination cost;
• decision;
• actual child count;
• actual quality delta;
• actual cost delta.

split_precision is the proportion of authorized splits that improve the registered quality target enough to justify their disclosed cost.

split_recall is the proportion of tasks for which the registered oracle or post-hoc analysis indicates that a beneficial split was available and the trigger selected it.

The benchmark must not tune the trigger on held-out rows.

Baseline discipline

Use the strongest sensible baseline for the exact task and information budget. Pin:

• model and revision;
• tokenizer;
• quantization;
• runtime;
• context and output limits;
• temperature and sampling;
• prompts and phase contracts;
• tool availability;
• hardware;
• warm or cold cache state;
• task corpus digest;
• completion-test implementation.

All retrieval, state compilation, validation, verification, repair, and routing calls are part of the system cost.

Run receipt

Every run emits:

run_id
suite_id
suite_version
commit
configuration_digest
corpus_or_task_digest
model_and_runtime
environment_digest
started_at
finished_at
raw_artifact_refs[]
metric_records[]
invalid_conditions_checked[]
known_limitations[]
result_state
receipt_digest

Allowed result states:

REPRODUCED
VERIFIED_MECHANISM
IMPLEMENTED
SOURCE_REVIEWED
SPECIFIED
HYPOTHESIS
INCONCLUSIVE
FAILED
AWAITING_LOCAL_RUN
INVALID

Statistical reporting

Preserve row-level data. For repeated measurements, report sample count, central tendency, dispersion, and uncertainty appropriate to the design.

Human evaluation must:

• use a frozen rubric;
• blind condition labels;
• randomize presentation order;
• retain disagreement;
• remain secondary to deterministic correctness where a test exists.

Negative evidence

Failed, invalid, and inconclusive runs remain in the ledger. Later success does not delete them.

A valid failure may show that:

• phase partitioning adds cost without improving quality;
• typed state improves auditability but not task success;
• recursive execution helps only weakly coupled tasks;
• stronger single-pass models dominate at the relevant budget;
• the recursion trigger cannot reliably predict beneficial decomposition.

Publication requirements

A public result page must show:

• exact evidence state;
• task corpus and split identity;
• model and runtime configuration;
• phase prompts and state schemas;
• resource accounting;
• row-level metrics;
• baseline comparison;
• failure gallery;
• evidence-recovery traces;
• invalid-run checks;
• limitations and non-claims;
• reproduction command;
• receipt digest.

No aggregate may be published without its raw rows and receipts.