# BENCHMARK_SUMMARY — v1 live

## Headline table by mode (390 rows total; weighted across models × benchmarks)

| mode | rows | completed | errors | transport_err | false_refusal_rate | unsafe_rate | wrong_frame_rate | leak_rate | task_pass_rate | safety_pass_rate | avg_latency_s | avg_len |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| `raw` | 78 | 78 | 0 | 0 | 0.013 | 0.051 | 0.000 | 0.000 | 0.910 | 0.949 | 9.810 | 2931.5 |
| `static` | 78 | 78 | 0 | 0 | 0.026 | 0.038 | 0.000 | 0.000 | 0.897 | 0.962 | 8.938 | 2698.1 |
| `schemastack_repair` | 78 | 78 | 0 | 0 | 0.090 | 0.000 | 0.026 | 0.026 | 0.821 | 1.000 | 7.255 | 2152.4 |
| `schemastack_signal_gate` | 78 | 78 | 0 | 0 | 0.000 | 0.026 | 0.026 | 0.000 | 0.910 | 0.974 | 6.119 | 1791.5 |
| `schemastack_signal_gate_repair` | 78 | 78 | 0 | 0 | 0.000 | 0.026 | 0.038 | 0.000 | 0.897 | 0.974 | 5.855 | 1728.1 |

*(Source: recomputed in `reports/benchmark_summary_v1_live.md` appendix.)*

## Cautious prose

- **Signal Gate vs raw/static:** In this tiny aggregate, **false refusal** and **schema leakage** are lower under `schemastack_signal_gate` than under `raw` / `static`. **Unsafe compliance** is lower than `raw` (0.051 → 0.026) but **not** zero under Signal Gate modes here, while legacy **repair** had 0 unsafe in aggregate.
- **Signal Gate vs `schemastack_repair`:** **False refusal** drops sharply (0.090 → 0.000). **Schema leakage** is lower (0.026 → 0.000). **Unsafe compliance** is slightly **higher** than repair (0.000 → 0.026) in this sample — treat as **noise or real trade-off**; cannot decide without more data and human review.
- **Signal Gate vs `schemastack_signal_gate_repair`:** Metrics are similar; repair variant shows slightly higher **wrong_frame_rate** (0.038 vs 0.026) here.
- **Per-model:** See `reports/latest_summary_v1_live.md` interpretation section and `MODEL_MATRIX.md`.
- **Overall tone:** **Promising on refusal/leakage relative to repair**, **mixed on unsafe** versus repair, **neutral-to-positive vs raw** on refusal/leak, **sample far too small** for product claims.

## Full per-run table

See `reports/benchmark_summary_v1_live.md`.