Skip to main content

​
5 — Data pipeline & ingestion — end-to-end flow, event-driven triggers & document intelligence.

This section explains how application data flows into the Layerup AI Agent, how the agent processes that data, and how it surfaces structured intelligence back to your systems of record. The entire pipeline is event-driven, integrates with your existing cloud-native infrastructure, and at no point requires data to leave your network boundary.

​
5.1 End-to-end data flow — the full lifecycle of a single case

The following describes the complete lifecycle of a single application case as it flows through the Layerup agent in an AWS environment. The Azure equivalent is structurally identical, with Azure service equivalents substituted at each stage.
stagedescription
Step 1 — IntakeAn application packet (PDFs, medical records, Rx history, APS documents, financial statements) arrives at your existing intake system. The packet is stored in the designated S3 input bucket and an intake metadata record is written to an SQS message.
Step 2 — TriggerAn EventBridge rule (or the SQS trigger directly) detects the new SQS message and triggers the Layerup agent container execution, passing the S3 object key(s) as parameters.
Step 3 — Document RetrievalThe agent’s container reads the application documents from S3 using its scoped IAM role. Documents never leave your S3 environment; the agent pulls them into its isolated execution memory.
Step 4 — Ingestion & NormalisationThe agent’s ingestion layer processes each document: PDFs are parsed (including scanned / OCR documents), data is extracted, normalised against the underwriting configuration schema, and structured for reasoning.
Step 5 — LLM Reasoning (via Bedrock)The agent constructs prompts against its configured Agent Operating Procedure (AOP) and invokes the designated foundation model via Amazon Bedrock’s private VPC endpoint. Bedrock Guardrails are applied at this layer.
Step 6 — Structured Output GenerationThe agent assembles its final structured output: the underwriting recommendation, confidence score, flagged inconsistencies, evidence citations, open questions, and requirement list.
Step 7 — Write-BackThe output JSON payload is written to your designated S3 output bucket. Simultaneously, a completion event is written to SQS / EventBridge, which your downstream system (policy admin, CRM) consumes to update the case status.
Step 8 — Audit LoggingAll agent reasoning steps, tool invocations, source citations, and model responses are written to CloudWatch Logs in structured JSON format for audit and compliance review.
Fig. A5.1 — End-to-end data flow. Every stage executes within your network boundary. No data transits to Layerup’s infrastructure at any point.

​
5.2 Ingestion mechanisms — event-driven, no polling required

The Layerup agent is triggered by events within your existing event infrastructure. No polling mechanism is required. Supported trigger patterns include:

​
5.2.1 Event-driven architecture

AWS SQS + S3 Event Notifications

When a new application packet is uploaded to the designated S3 input bucket, S3 publishes an event notification to an SQS queue. The agent’s ECS Service or AgentCore Runtime is configured with an SQS trigger that automatically starts a new agent task for each message. This is the recommended trigger pattern for most workloads.

AWS EventBridge

For more complex routing logic (e.g., routing different application types to different agent configurations or versions), EventBridge Rules filter S3 events by prefix, metadata tag, or application type, and route each to the appropriate agent version. Useful when multiple agent configurations handle distinct case types.

API Gateway (Synchronous)

If your existing workflow system requires synchronous invocation (i.e., it calls an endpoint and waits for a response), an API Gateway endpoint can be placed in front of the agent, with a Lambda function handling the async-to-sync bridge. Given typical processing times of 7–55 minutes, the asynchronous pattern with a callback webhook is generally preferred for production workloads.

Azure Service Bus + Blob Trigger

On Azure, Blob Storage triggers an Azure Service Bus message on new document upload. Azure Container Apps’ KEDA (Kubernetes Event-Driven Autoscaling) consumes the Service Bus queue and scales agent container instances accordingly. Structurally equivalent to the SQS + S3 Event Notifications pattern on AWS.

​
5.2.2 Handling unstructured and poor-quality documents

A core capability of the Layerup AI Agents is the ability to ingest and extract meaningful intelligence from the imperfect document formats endemic to the insurance industry. Multi-pass OCR pipeline — Scanned PDFs and handwritten documents are processed through a multi-pass OCR pipeline before LLM reasoning. The OCR output is normalised and confidence-scored before being passed to the reasoning layer. Low-confidence OCR extractions are flagged as uncertain in the agent’s evidence citations — the agent does not silently propagate low-quality extractions into its reasoning. Multi-document context assembly — The agent’s ingestion layer reads across multiple documents simultaneously (application form, APS, Rx history, financial statements, attending physician statements) and constructs a unified case context. Cross-document inconsistencies — for example, a reported occupation on the application form that contradicts the duties described in a supporting job letter — are surfaced proactively in the agent’s output. Third-party data feed integration — Structured data from licensed third-party providers (Milliman, pharmacy claims databases, MIB) can be integrated via direct API connector or scheduled file drop, depending on the provider’s delivery mechanism. The agent’s ingestion layer normalises these feeds into its unified case schema alongside the unstructured documents from your intake system. Ambiguity flagging — When document quality or clarity prevents confident extraction, the agent flags the specific field as uncertain rather than hallucinating a value. These flags are surfaced in the requirements and evidence citation layers of the output payload, directing the human underwriter to the specific document and field requiring manual review. Fig. A5.2 — Ingestion layer architecture. Every document type — structured or unstructured, clean or degraded — passes through the same normalisation and ambiguity-flagging pipeline before entering the reasoning layer.