Stable context is a runtime resource
Compile the reusable decision prefix once instead of re-evaluating it for every state transition.
DecisioExperimental← Architecture & Infrastructure
STATEFUL DECISIONS · LOCAL LLM
Decisio
Make repeated local-LLM decisions without re-prefilling the same stable context every time.
STABLE CONTEXTTask + policy + constraintsPrefill once
State 1 → actionState 2 → actionState 3 → action
DECIDE
Repeated decisions should not pay for the same long prefix every time.
Decisio targets bounded decision loops where the task, policy and part of the prompt stay stable while only the current state changes.
Constraints stay deterministic
Impossible actions and known rules are filtered in software before the model is asked to choose.
The output is an action
For bounded choices, read candidate logits directly and return a typed decision instead of generating prose.
BUILD
Split what stays stable from what changes.
The runtime keeps prefix-safe model state reusable, appends only the current-state suffix and scores the currently valid candidates.
STABLETask + policyReusable model state
DYNAMICCurrent stateOnly what changed
CONSTRAINValid candidatesDeterministic filtering
READOUTTyped actionZero answer tokens
TEST
Use workloads where repeated state actually matters.
Snake is the visual reference loop, while routing, triage and policy gates represent the broader class of bounded decisions Decisio is designed for.
Control loop
Stable objective and sensor semantics; changing board state; typed move as output.
Support queue
Stable routing policy and queue meanings; new ticket state; queue ID as the action.
Decision gate
Stable policy semantics; changing request; allow, deny or escalate as the typed result.
MEASURE
Reuse only matters if it preserves the fresh decision.
The benchmark contract measures both efficiency and correctness: less physical model work is only valuable when the reused path stays equivalent to the fresh path.
EVIDENCE BOUNDARY
The evidence contract compares state reuse against a fresh oracle.
Decisio records choice equivalence, score deltas, logical versus physically evaluated tokens, reused prefix tokens, cache hits and misses, latency and bounded context-state bytes.
Supported today
- Stable-context model-state reuse is implemented
- Fresh-path correctness checks are part of the runtime contract
- Direct A/B/C choice readout generates zero answer tokens on the native path
- Exact model/runtime provenance is recorded
Not claimed here
- Representative release-grade evidence across a broader workload set
- Calibrated confidence / answerability contract
DECIDE AGAIN
Use Decisio only where the repeated-state structure is real.
The architecture is useful when a meaningful prefix can be reused and the output is a bounded action. It is not intended for unrelated requests or free-form generation.
CURRENT DECISION
Keep the stable decision contract warm. Recompute only what changed.
The product thesis is not generic caching. It is a stateful decision boundary with explicit constraints, prefix-safe reuse and typed outputs.
- Good fit: repeated bounded decisions with stable policy/context
- Good fit: local GGUF workloads where repeated prefill is material
- Poor fit: unrelated prompts or genuinely free-form output