Objective
Establish a reproducible, falsifiable evidence path for p03-fovea-address-fabric. This contract authorizes no result by itself. A public result requires a versioned run, exact configuration, raw artifacts, environment identity, limitations, and a receipt.
Required suites
- Address golden vectors
- Parent/child/neighbor property tests
- Hilbert encode/decode and locality
- LOOK determinism and restore
- Atlas deletion and reconstruction
- Proof-breach rendering
- Bytes/tokens moved versus competent lazy loading
- Landmarked field versus plain grid versus search
Invalid-run conditions
A run is invalid, rather than merely low-scoring, when any of the following occurs:
- incorrect address arithmetic
- address churn
- trusted unfold without receipt
- projection cache treated as truth
- unreceipted benchmark configuration
Metrics
Report each metric separately. Do not hide a regression inside one composite score.
exactness
reconstruction agreement
bytes_moved
tokens_moved
time_to_refind
navigation_errors
peripheral_cue_use
user_preference
Baseline discipline
Use the strongest sensible baseline for the exact task and information budget. Pin versions, prompts, corpora, models, hardware, runtime flags, and warm/cold state. If the research system receives additional context, dependencies, precomputation, or a model, charge or disclose them under the contract.
Run receipt
Every run emits:
run_id
suite_id
suite_version
commit
configuration_digest
corpus_or_task_digest
model_and_runtime
environment_digest
started_at
finished_at
raw_artifact_refs[]
metric_records[]
invalid_conditions_checked[]
known_limitations[]
result_state
receipt_digest
Allowed result states:
REPRODUCED
VERIFIED_MECHANISM
IMPLEMENTED
SOURCE_REVIEWED
SPECIFIED
HYPOTHESIS
INCONCLUSIVE
FAILED
AWAITING_LOCAL_RUN
INVALID
Data splits and leakage
Freeze task sets before viewing final results. Keep development, selection, and held-out evaluation separate where the suite supports them. Record any human-authored fixture that appeared during implementation. Do not tune a threshold on the final set and present it as held out.
Statistical and qualitative reporting
For repeated measurements, report sample count, central tendency, dispersion, and uncertainty appropriate to the design. Preserve row-level results. For human ratings, blind condition labels where possible, record the rubric, and retain disagreement rather than averaging away failure modes.
Negative evidence
Failed and inconclusive runs remain in the experiment ledger with code, configuration, and reason. A later successful run does not delete them.
Publication requirements
A public result page must show:
- exact evidence state;
- date and commit;
- environment and task/corpus identity;
- metric table;
- baseline table;
- raw artifact links;
- failure gallery;
- limitations and non-claims;
- reproduction command;
- receipt digest.
No marketing page may round a measured result into a broader claim than the contract supports.