7 — Observability, logging & audit trails — telemetry architecture, deterministic reasoning capture & regulatory compliance.

This section addresses the requirements of your internal audit, compliance, and IT operations teams. The Layerup agent is designed to be a fully transparent workload — every decision, reasoning step, document extraction, and model inference is logged in structured, queryable format within your own observability infrastructure. No telemetry is emitted to Layerup’s systems.

7.1 Telemetry architecture — structured, queryable, yours

The Layerup agent emits structured telemetry to your native cloud monitoring systems throughout the lifecycle of each agent session. The telemetry architecture is designed to be immediately consumable by your existing operations and audit tooling without bespoke integration.

7.1.1 AWS: Amazon CloudWatch

The agent emits the following data streams to CloudWatch:

log stream	level	contents
Application Logs	`INFO`	High-level session events: session start, document ingestion complete, LLM reasoning initiated, reasoning complete, output written, session end. Useful for operational dashboards and SLA monitoring.
Audit Logs	`AUDIT`	Immutable, append-only structured log entries for each agent decision, source citation, and flag raised. Includes the full reasoning chain that produced each recommendation, enabling complete reconstruction of the agent’s decision process for any case.
Error Logs	`ERROR` / `WARN`	Structured error events for failed document reads, OCR failures, model timeout events, and unexpected exceptions. Each event carries a correlation ID linking it back to the originating case.
Metrics	CloudWatch Metrics	Custom metrics: `cases_processed_count`, `cases_flagged_for_escalation_count`, `average_processing_time_seconds`, `document_ocr_confidence_score_average`, `guardrail_interventions_count`. Powers operational dashboards and drives CloudWatch Alarms.

7.1.2 Azure: Azure Monitor and Application Insights

On Azure, equivalent telemetry flows to Azure Monitor and Application Insights:

Structured logs are emitted to a Log Analytics Workspace, enabling KQL queries across all agent sessions and cross-session analytics.
Azure Application Insights provides distributed tracing across the agent’s processing pipeline, enabling your team to visualise the end-to-end processing timeline for any case.
Azure Monitor Alerts are configured to notify your on-call operations team of any agent health degradation or SLA breach.

7.2 Deterministic audit trails — regulatory-grade reasoning capture

A critical requirement for any regulated carrier deploying AI in underwriting decisions is the ability to produce a complete, immutable audit trail for any given case — including the AI’s reasoning — to satisfy state insurance department examination requirements, internal audit standards, and potential legal discovery.

7.2.1 Step-level reasoning capture

For every underwriting case, the Layerup agent writes a structured audit record that contains:

Prompt Transparency

The exact prompt sent to the foundation model for each reasoning step (with applicant PII redacted according to your Bedrock Guardrail configuration). No reasoning occurs in a black box.

Model Response Capture

The model’s complete response for each reasoning step, including the specific guardrail policy evaluations applied and their outcomes.

Decision Logic Trace

The agent’s decision logic at each step: which rules from the AOP were triggered, what evidence was evaluated, and what conclusion was reached — mapped to specific AOP rule identifiers.

Source Citations

Source citations for every extracted data point, including the document filename, page number, section, and verbatim extracted text excerpt.

Millisecond Timestamps

Timestamps with millisecond precision for each reasoning step — enabling point-in-time reconstruction of the agent’s processing timeline for any case.

Version Pinning

The AOP version and model version active at the time of the case — enabling exact point-in-time reconstruction of the agent’s configuration and behavior for historical review or re-examination.

7.2.2 CloudWatch Log Insights — example audit queries

Your audit team uses CloudWatch Log Insights to query the structured audit trail. The structured log format supports a wide range of compliance queries without requiring custom tooling:

-- Retrieve the full reasoning chain for a specific case
fields @timestamp, reasoning_step, aop_rule_triggered, evidence_cited, conclusion
| filter case_id = "case_20260601_00123"
| sort @timestamp asc

-- Find all cases where a Bedrock Guardrail was triggered in the last 30 days
fields @timestamp, case_id, guardrail_policy, guardrail_outcome
| filter guardrail_intervention = true
| filter @timestamp > dateadd(day, -30, now())

-- Identify low-confidence cases where recommendation was Defer
fields @timestamp, case_id, confidence_score, ai_recommendation
| filter confidence_score < 85
| filter ai_recommendation = "Defer Pending Requirements"
| stats count(*) by bin(1d)

-- Count escalations to senior underwriter by occupation class and month
fields @timestamp, case_id, occupation_class, escalation_reason
| filter escalation_flag = true
| stats count(*) by occupation_class, datefloor(@timestamp, 1mo)

7.2.3 Log retention and immutability

CloudWatch Log Groups containing agent audit logs are configured to meet insurance regulatory record-keeping requirements:

control	configuration
Retention Policy	Minimum 7-year retention, aligned with insurance regulatory record-keeping requirements. Configurable to match your organization’s own retention schedule.
Log Group Lock (Data Protection)	Your team applies a CloudWatch Logs data protection policy to prevent log tampering or deletion. Audit logs are append-only from the moment of creation.
Export to S3 and Glacier	Long-term audit logs are exported from CloudWatch to S3 and transitioned to S3 Glacier Deep Archive for cost-effective long-term retention, encrypted with your KMS CMK.
Azure Equivalent	Log Analytics Workspace with 7-year retention, immutability policy enabled, and Azure Monitor export to Azure Storage with lifecycle management to archive tier.

Fig. A7.1 — Audit log lifecycle. From step-level emission to Glacier archival, every audit record remains within your account, under your encryption keys, with tamper prevention applied from the moment of creation.

Foundations

Deployment Architecture

Security & Governance

Data & Integration

Observability & Resilience

Configuration & Lifecycle

Confidence & Scoring

Reference

Observability, Logging & Audit Trails

7 — Observability, logging & audit trails — telemetry architecture, deterministic reasoning capture & regulatory compliance.

7.1 Telemetry architecture — structured, queryable, yours

7.1.1 AWS: Amazon CloudWatch

7.1.2 Azure: Azure Monitor and Application Insights

7.2 Deterministic audit trails — regulatory-grade reasoning capture

7.2.1 Step-level reasoning capture

Prompt Transparency

Model Response Capture

Decision Logic Trace

Source Citations

Millisecond Timestamps

Version Pinning

7.2.2 CloudWatch Log Insights — example audit queries

7.2.3 Log retention and immutability

​7 — Observability, logging & audit trails — telemetry architecture, deterministic reasoning capture & regulatory compliance.

​7.1 Telemetry architecture — structured, queryable, yours

​7.1.1 AWS: Amazon CloudWatch

​7.1.2 Azure: Azure Monitor and Application Insights

​7.2 Deterministic audit trails — regulatory-grade reasoning capture

​7.2.1 Step-level reasoning capture

Prompt Transparency

Model Response Capture

Decision Logic Trace

Source Citations

Millisecond Timestamps

Version Pinning

​7.2.2 CloudWatch Log Insights — example audit queries

​7.2.3 Log retention and immutability

7 — Observability, logging & audit trails — telemetry architecture, deterministic reasoning capture & regulatory compliance.

7.1 Telemetry architecture — structured, queryable, yours

7.1.1 AWS: Amazon CloudWatch

7.1.2 Azure: Azure Monitor and Application Insights

7.2 Deterministic audit trails — regulatory-grade reasoning capture

7.2.1 Step-level reasoning capture

7.2.2 CloudWatch Log Insights — example audit queries

7.2.3 Log retention and immutability