> ## Documentation Index
> Fetch the complete documentation index at: https://docs.uselayerup.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Versioning, Release Gates & Failure Modes

> Releases are configuration promotions across a fixed environment ladder, gated by evals, drift, and error budgets. Every plane has a documented failure-mode matrix; failure is a first-class state.

# 21 — Versioning, release gates & failure modes.

Layerup releases are configuration promotions across a fixed environment ladder, gated by
evals, drift, and error budgets. Every plane has a documented failure-mode matrix; failure
is a first-class state of the substrate.

## 21.1  Version domains

| Domain           | Versioning                   | Pin location           |
| ---------------- | ---------------------------- | ---------------------- |
| Ontology         | release line + branches (§6) | agent / tool / mapping |
| Mapping          | semver per mapping           | tenant config          |
| Tool             | semver per signature         | agent definition       |
| Prompt           | semver hash                  | agent / tool           |
| Retrieval corpus | snapshot id                  | tool / agent           |
| Model            | id + lineage (§12)           | routing policy         |
| Agent            | semver per definition        | tenant config          |
| Policy           | policy version (§16.6)       | tenant                 |

## 21.2  Promotion ladder

```mermaid theme={null}
flowchart TB
  D[dev] --> S[staging]
  S --> U[UAT]
  U --> SH[shadow / canary]
  SH --> P[production]
  P --> R[rollback]
  R -.-> P
  G1{eval gate} -.- D
  G2{eval + replay} -.- S
  G3{tenant UAT sign-off} -.- U
  G4{drift + error budget} -.- SH
```

*Fig. 21.1 — Promotion ladder. Gates run automatically at each step; manual sign-off is required at UAT. Rollback is one operator action.*

## 21.3  Release gate matrix

| Gate            | Inputs                        | Pass condition                          | Block action                                      |
| --------------- | ----------------------------- | --------------------------------------- | ------------------------------------------------- |
| Schema          | artefact diff vs trunk        | semver matches change class (§6.1)      | auto-block                                        |
| Eval            | candidate vs baseline         | bench ≥ pack.min, regression ≤ pack.max | auto-block                                        |
| Replay          | recent runs sample            | verdict-delta within noise band         | auto-block · operator override possible with note |
| Adversarial     | probe pack                    | no regression on probes                 | auto-block                                        |
| Drift           | trailing window               | no ≥ 2σ warn                            | operator alert · block at 3σ                      |
| Error budget    | SLO burn-rate                 | no fast-burn breach                     | auto-block                                        |
| Tenant sign-off | UAT outcome                   | tenant-approved                         | auto-block                                        |
| Security review | marking change · scope change | security primary signs                  | auto-block                                        |

## 21.4  Canary algorithm

1. Route a small percentage of eligible traffic to the candidate (default 1%).
2. Observe per-step quality, latency, error budget, and drift sigma in 15-minute windows.
3. Ramp to 5% → 25% → 100% only if all watched metrics stay within bounds.
4. Any 2σ deviation pauses the ramp; any 3σ deviation rolls back.
5. Rollback is content-hash promotion of the prior version; takes effect within one minute.

## 21.5  Rollback strategy

* Every promoted artefact records its predecessor's content hash; rollback is a one-step re-promotion of the predecessor.
* In-flight runs at rollback continue against their pinned versions; no run is mid-air-upgraded.
* Rollbacks are typed AuditEvents (`config.rollback`) on the tenant chain.

## 21.6  Failure-mode matrix

The platform classifies failures into 16 typed modes. Each mode has a detection rule, a containment path, a fallback, and an audit signature. This list is canonical; tenant-specific modes inherit from these classes.

| #  | Mode                         | Detection                                                                | Containment                                 | Fallback                                              | Audit                              |
| -- | ---------------------------- | ------------------------------------------------------------------------ | ------------------------------------------- | ----------------------------------------------------- | ---------------------------------- |
| 1  | Hallucination                | verifier rule pack · evidence missing for asserted property              | verifier blocks Decision · raises Exception | handoff to reviewer queue                             | `verify.block.hallucination`       |
| 2  | Schema mismatch              | JSON-Schema validation fails at ingest or tool boundary                  | route to quarantine queue                   | typed Exception · operator triage                     | `data.ingest.schema_drift`         |
| 3  | Stale data                   | SoR version observed \< current at commit time                           | refuse commit · re-resolve                  | re-stage with new pre-image                           | `action.stale_target`              |
| 4  | Source-of-record downtime    | adapter error rate window · circuit breaker                              | open breaker · pause adapter                | flat-file path or queue retry                         | `integration.source_down`          |
| 5  | Ambiguous evidence           | conflicting EvidenceSpans on same Property                               | verifier emits Exception                    | handoff with rationale                                | `evidence.ambiguous`               |
| 6  | Conflicting evidence         | cross-document contradiction detected                                    | verifier blocks · Exception                 | handoff · request operator clarification              | `evidence.conflict`                |
| 7  | Low-confidence extraction    | extractor confidence \< agent threshold                                  | verifier classifies as warn or block        | handoff or alternate extractor                        | `extract.low_confidence`           |
| 8  | Model drift                  | drift sigma breach (§13.4)                                               | demote to shadow / demoted                  | route via fallback model                              | `model.drift.breach`               |
| 9  | Privilege escalation attempt | scope check fails on dispatch                                            | PDP denies · principal flagged              | raise security alert · lock if pattern                | `policy.deny.privilege_escalation` |
| 10 | Unauthorised tool invocation | marking or purpose mismatch                                              | PDP denies                                  | operator alert · audit only                           | `policy.deny.purpose`              |
| 11 | Duplicate effect             | idempotency-key collision with different payload                         | refuse second commit · raise Exception      | operator review · explicit supersession only          | `action.duplicate`                 |
| 12 | Adapter writeback failure    | adapter returns error after exhausted retries                            | action moves to `failed`                    | compensation or replay after fix                      | `action.adapter_error`             |
| 13 | RAG poisoning                | retrieved spans inconsistent with ontology types · adversarial probe hit | verifier blocks · corpus quarantined        | fallback corpus · re-index                            | `retrieval.poisoned`               |
| 14 | Prompt injection             | injection-detector classifier on input · canary tokens                   | refuse to dispatch reasoning step           | handoff · reviewer triage                             | `reasoning.injection`              |
| 15 | Authority breach             | authority predicate fails at PDP or commit                               | refuse stage / refuse commit                | handoff to higher authority                           | `authority.breach`                 |
| 16 | Audit chain breach           | chain verifier finds hash mismatch                                       | seal chain at breach point                  | start new chain · operator + tenant security incident | `audit.tamper_detected`            |

## 21.7  Incident response posture

* Severity bands (S0–S3) per failure mode with documented response times.
* Tenant security primary notified on S0 / S1 and on any audit-chain breach within 15 minutes.
* Postmortems include contributory drift / eval signals and any release that crossed the gate.
* Postmortems become typed AuditEvents (`incident.summary`) on the tenant chain.

## 21.8  Change freeze

During declared change freezes (tenant-defined windows or platform-declared post-incident windows), promotion is auto-blocked across all domains except security-critical patches. Freeze status is visible on the tenant config dashboard.