Documentation Index
Fetch the complete documentation index at: https://docs.uselayerup.com/llms.txt
Use this file to discover all available pages before exploring further.
21 — Versioning, release gates & failure modes.
Layerup releases are configuration promotions across a fixed environment ladder, gated by evals, drift, and error budgets. Every plane has a documented failure-mode matrix; failure is a first-class state of the substrate.21.1 Version domains
| Domain | Versioning | Pin location |
|---|---|---|
| Ontology | release line + branches (§6) | agent / tool / mapping |
| Mapping | semver per mapping | tenant config |
| Tool | semver per signature | agent definition |
| Prompt | semver hash | agent / tool |
| Retrieval corpus | snapshot id | tool / agent |
| Model | id + lineage (§12) | routing policy |
| Agent | semver per definition | tenant config |
| Policy | policy version (§16.6) | tenant |
21.2 Promotion ladder
Fig. 21.1 — Promotion ladder. Gates run automatically at each step; manual sign-off is required at UAT. Rollback is one operator action.21.3 Release gate matrix
| Gate | Inputs | Pass condition | Block action |
|---|---|---|---|
| Schema | artefact diff vs trunk | semver matches change class (§6.1) | auto-block |
| Eval | candidate vs baseline | bench ≥ pack.min, regression ≤ pack.max | auto-block |
| Replay | recent runs sample | verdict-delta within noise band | auto-block · operator override possible with note |
| Adversarial | probe pack | no regression on probes | auto-block |
| Drift | trailing window | no ≥ 2σ warn | operator alert · block at 3σ |
| Error budget | SLO burn-rate | no fast-burn breach | auto-block |
| Tenant sign-off | UAT outcome | tenant-approved | auto-block |
| Security review | marking change · scope change | security primary signs | auto-block |
21.4 Canary algorithm
- Route a small percentage of eligible traffic to the candidate (default 1%).
- Observe per-step quality, latency, error budget, and drift sigma in 15-minute windows.
- Ramp to 5% → 25% → 100% only if all watched metrics stay within bounds.
- Any 2σ deviation pauses the ramp; any 3σ deviation rolls back.
- Rollback is content-hash promotion of the prior version; takes effect within one minute.
21.5 Rollback strategy
- Every promoted artefact records its predecessor’s content hash; rollback is a one-step re-promotion of the predecessor.
- In-flight runs at rollback continue against their pinned versions; no run is mid-air-upgraded.
- Rollbacks are typed AuditEvents (
config.rollback) on the tenant chain.
21.6 Failure-mode matrix
The platform classifies failures into 16 typed modes. Each mode has a detection rule, a containment path, a fallback, and an audit signature. This list is canonical; tenant-specific modes inherit from these classes.| # | Mode | Detection | Containment | Fallback | Audit |
|---|---|---|---|---|---|
| 1 | Hallucination | verifier rule pack · evidence missing for asserted property | verifier blocks Decision · raises Exception | handoff to reviewer queue | verify.block.hallucination |
| 2 | Schema mismatch | JSON-Schema validation fails at ingest or tool boundary | route to quarantine queue | typed Exception · operator triage | data.ingest.schema_drift |
| 3 | Stale data | SoR version observed < current at commit time | refuse commit · re-resolve | re-stage with new pre-image | action.stale_target |
| 4 | Source-of-record downtime | adapter error rate window · circuit breaker | open breaker · pause adapter | flat-file path or queue retry | integration.source_down |
| 5 | Ambiguous evidence | conflicting EvidenceSpans on same Property | verifier emits Exception | handoff with rationale | evidence.ambiguous |
| 6 | Conflicting evidence | cross-document contradiction detected | verifier blocks · Exception | handoff · request operator clarification | evidence.conflict |
| 7 | Low-confidence extraction | extractor confidence < agent threshold | verifier classifies as warn or block | handoff or alternate extractor | extract.low_confidence |
| 8 | Model drift | drift sigma breach (§13.4) | demote to shadow / demoted | route via fallback model | model.drift.breach |
| 9 | Privilege escalation attempt | scope check fails on dispatch | PDP denies · principal flagged | raise security alert · lock if pattern | policy.deny.privilege_escalation |
| 10 | Unauthorised tool invocation | marking or purpose mismatch | PDP denies | operator alert · audit only | policy.deny.purpose |
| 11 | Duplicate effect | idempotency-key collision with different payload | refuse second commit · raise Exception | operator review · explicit supersession only | action.duplicate |
| 12 | Adapter writeback failure | adapter returns error after exhausted retries | action moves to failed | compensation or replay after fix | action.adapter_error |
| 13 | RAG poisoning | retrieved spans inconsistent with ontology types · adversarial probe hit | verifier blocks · corpus quarantined | fallback corpus · re-index | retrieval.poisoned |
| 14 | Prompt injection | injection-detector classifier on input · canary tokens | refuse to dispatch reasoning step | handoff · reviewer triage | reasoning.injection |
| 15 | Authority breach | authority predicate fails at PDP or commit | refuse stage / refuse commit | handoff to higher authority | authority.breach |
| 16 | Audit chain breach | chain verifier finds hash mismatch | seal chain at breach point | start new chain · operator + tenant security incident | audit.tamper_detected |
21.7 Incident response posture
- Severity bands (S0–S3) per failure mode with documented response times.
- Tenant security primary notified on S0 / S1 and on any audit-chain breach within 15 minutes.
- Postmortems include contributory drift / eval signals and any release that crossed the gate.
- Postmortems become typed AuditEvents (
incident.summary) on the tenant chain.