Cache-Aware Prompt Layout
Stable prefixes, bounded evidence tails, and model-specific invalidation
A prompt should be organized by mutability.
Stable prefix
- system doctrine;
- output schemas;
- style rules;
- verification rules;
- tool contracts;
- project-wide accepted constraints.
Session-stable middle
- active Plan;
- surface structure;
- actor or project identity;
- current authority.
Volatile evidence tail
- source spans;
- current question;
- recent events;
- active frontier;
- exact output request.
This layout supports automatic prefix caching and model-specific KV artifacts where available.
Cache key
A cache artifact must include:
weights digest
tokenizer
runtime
attention implementation
precision
position profile
block size
exact span ContentIds
dependency root
A cache from one model or runtime cannot be injected into another without an explicit adapter and conformance benchmark.
Invalidation
Changes to policy, authority, schema, compiler, model, tokenizer, source span, or project base version invalidate affected cache regions.
Graceful degradation
When an API exposes no cache internals, the architecture still gains from exact result cache, semantic aliases, bounded context, and smaller outputs.
The target metric is end-to-end time to verified result, not cache hit rate by itself.
Source register
- N08-S01 — GlyphCAS Design Report.
How_we_can_make_CAS_lightning_fast_for_any_LLM_by_design_or_adaptation.docx - N08-S02 — Zeke 4B Max-Flex Runtime Master Spec.
ZEKE-4B-MAXFLEX-RUNTIME-MASTER-SPEC-v0.1.md - N08-S03 — LM Jefe Full System Specification.
LM_JEFE_FULL_SYSTEM_SPEC.md