An InsurTech platform needs the facts buried inside home paperwork - inspection reports, contractor invoices, equipment manuals, warranties - as structured data its systems can act on. We designed a RAG-based extraction pipeline that classifies every chunk, runs specialised agents against a fixed schema, links each issue to the asset it belongs to, and traces every extracted field back to the page it came from.
Orchestrated pipeline nodes, schema load to final output
Extraction agents across two interchangeable model paths
Deterministic supporting services, no LLM involved
Entity categories every document chunk is routed to
An InsurTech company working to reduce home-related risk for policyholders and give carriers portfolio visibility, fewer losses, and a route to proactive mitigation.
The facts that matter sit in unstructured home documents - inspection reports, invoices, manuals, warranties - in wildly different shapes, read by hand.
A RAG-based extraction pipeline on a LangGraph state machine: category-based retrieval, specialised extraction agents, then validation, sourcing and relationship resolution.
Four linked entity types - the home, its assets, the issues observed against them, and document metadata - schema-valid and traceable to the chunk each field came from.
The client is an InsurTech company built around a simple observation: most home insurance losses are preventable, and the signals that predict them are already written down. Their platform exists to reduce home-related risk for policyholders while giving carriers portfolio visibility, lower loss ratios, and a way to intervene before a claim rather than after one.
To do that, the platform needs to know what a home actually contains. How old is the roof and what is it made of. What is the make, model and serial number of the water heater, and when was it installed. Which issues did the last inspection flag, against which piece of equipment, and how urgent were they. What warranty covers the furnace, and until when.
All of that exists - in a 40-page inspection report, a contractor's invoice, an equipment manual, a warranty certificate. None of it exists as data. Every one of those documents was written for a human reader, by a different author, in a different format, with no shared vocabulary and no obligation to be consistent.
This engagement built the layer that closes that gap: a proof of concept that takes those documents as they are and returns a structured, validated, source-traced record of the home.
The extracted record has to satisfy a machine and a person at the same time - which is what forces both the schema and the audit trail.
A field with no provenance is unusable to both audiences. The carrier's systems can't reconcile it and the reviewer can't defend it - so traceability back to the source chunk was designed in from the first node, not bolted on at the end.
Handing a whole inspection report to a language model and asking for JSON gets you something that looks right and can't be trusted. The hard part isn't reading the document - it's doing it the same way every time, against a schema, with the relationships intact and every value attributable.
An inspection report, an invoice and a warranty share almost no structure. Section names, tables, headings and terminology vary by author, tool and trade - so no single parsing rule survives the second document.
One paragraph can describe the home, name an asset, and flag an issue against it. Pulling all of it in one pass blurs the distinction the schema depends on - and unit-level asset detail leaks into home properties.
A plausible model number, an invented install date, a cost that was never printed. In a record a carrier acts on, a confidently wrong field is worse than a missing one - and looks identical until someone checks.
"Corrosion at the base" only matters once it is attached to the water heater. An issue with no asset behind it, or an asset pointing at a document that doesn't exist, is noise dressed as structure.
The furnace appears in the inspection report, the invoice that serviced it and its own manual. Without a deduplication rule the record inflates - three furnaces where the home has one.
Reading a document set by hand takes an hour or more and produces different results depending on who read it. Coverage varies, the backlog grows, and the cost sits between the carrier and any proactive intervention.
Proactive mitigation is the whole product promise, and it depends entirely on knowing what a home contains and what condition it is in. You cannot act early on data you don't have in a shape you can query - so the extraction layer wasn't a convenience feature, it was the precondition for everything the platform is trying to do.
Focaloid engaged to design and build the extraction system as a proof of concept - trustworthy enough that its output could go into a review UI and, from there, into the platform's own data model. Three constraints were set as hard requirements rather than aspirations.
No free-form JSON, ever. Every extracted entity is validated against a centralised schema - required fields, data types, enums and value constraints, date and date-time formats, nested objects and arrays, and a check for properties that shouldn't be there at all.
Nothing enters the record without a source. Each entity carries the document, page and section it was read from, so any value in the UI can be opened back to the paragraph that produced it - by a reviewer or by an auditor.
The same document set produces the same record: zero-temperature inference with a fixed seed and version-controlled prompts. And a failure in one extractor degrades to an empty result for that entity rather than taking the whole run down.
One agent per entity type - not one prompt for the whole document. A single mega-prompt asked to return the home, its assets, its issues and its document metadata at once has to hold four different jobs in its head, and quality drops on all four. So the method was split: each extractor gets a narrow prompt, its own retrieval categories, and one entity type to be right about. The orchestrator's job is sequencing and consolidation, not reasoning.
Fix the four entity types and the JSON schemas they must satisfy, and define the chunk categories retrieval will route on.
Parse, chunk, classify and embed each document, then store the chunks in session-scoped vector collections with their metadata.
Build the LangGraph state machine, the specialised extractors, and the alternative unified path - with retries and graceful degradation.
Validate against schema, assign sources, resolve cross-entity references, clean the payload, and expose it all in the review UI.
The system extracts and structures. It deliberately does not score risk, price a policy, or write back into any system of record - the structured output lands in the database and the review UI, and a person confirms it. Equally deliberately, the proof of concept is bounded: four entity types, three file formats, one document set per session. What it refuses to infer is as much a design decision as what it extracts.
A user uploads a document set. Each file is parsed, chunked, classified and embedded into a session-scoped vector store. The orchestrator then walks a fixed sequence of nodes: retrieve the right context per entity, extract, merge, validate, source, resolve relationships - and either retry or finish.
The console replays a representative run: files are parsed and chunked, chunks are classified into the four categories, context is retrieved per entity, the specialised extractors run, and then validation, sourcing and relationship resolution complete the record before it is stored.
Illustrative run with representative data - in the delivered system every chunk, count and validation result is computed from the uploaded documents.
A React front end uploads documents and reviews results over a REST API. A FastAPI backend parses and chunks them, classifies each chunk, and stores embeddings in session-scoped Qdrant collections. The LangGraph orchestrator then drives extraction across either provider, and four deterministic services turn raw extraction into a governed record. PostgreSQL holds state and results; LangSmith traces every call.
Extraction has real branching: two provider paths, a node that must wait for another to finish, and a completion check that can send the run back for a retry. A LangGraph StateGraph makes that explicit - shared state, one node per job, checkpointed so a run can be recovered rather than restarted - and it means every step is a place to gate, log and trace.
The as-scoped system - redrawn from the engagement's architecture documentation, anonymised. Runs under Docker Compose in local and production configurations.
The orchestrator executes these in order, holding shared state (IngestionState) across all eight.
Load and parse the JSON schema definitions every downstream node validates against.
Pull entity-specific context from Qdrant for all four entity types at once, deduplicated by document and chunk index.
Run the specialised extractors in parallel on the OpenAI path, or the single unified extractor on the Claude path.
Merge the outputs and deduplicate assets, issues and documents by signature.
Validate every entity against its schema - required fields, types, enums, date formats, nested arrays.
Attach the document, page and section metadata that makes each entity traceable to its chunk.
Validate cross-entity references - issues to assets, assets to issues and documents - and strip the invalid ones.
The conditional gate: complete the run, or route back through extraction for another attempt.
Anyone can prompt a model for JSON. The engineering that makes an extracted record safe to store, review and act on lives in a handful of deliberate choices - each one below is as implemented in the proof of concept.
The naive approach hands every extractor the whole document. That buries the relevant paragraph in noise, burns context, and makes the asset extractor reason about roof material. So every chunk is classified into one of four categories at ingestion time, and each extractor retrieves only from the categories that matter to it.
Rule-based keyword matching takes priority and handles the clear majority; the LLM is a fallback for genuinely ambiguous chunks only, batched for throughput. The priority order is deliberate - issues are the highest-signal category, home characteristics the hardest to claim on a single keyword.
Retrieval is category-scoped rather than global, and runs in parallel for all four entity types. Chunks are deduplicated by document and chunk index, then aggregated and ranked, and the context string carries its own metadata forward - which is what makes source assignment possible later.
The asset extractor also reads the issues category. Equipment is frequently named for the first time inside a defect note - "rust at the base of the 2016 Rheem water heater" is both an issue and the only place that asset appears. Retrieving issues for assets is what stops those assets going missing.
The system supports both OpenAI and Claude, selected per run by a single model_provider parameter. This isn't hedging - the two paths have genuinely different shapes, and having both in the proof of concept is how the trade-off gets measured on the client's own documents rather than argued about.
| OpenAI path | Claude path | |
|---|---|---|
| Shape | Four specialised extractors | One unified extractor |
| Execution | Parallel, with issues sequenced after assets | All entity types in a single API call |
| Prompt | Narrow, one entity type each | All context types combined into one prompt |
| Large documents | MAP_MERGE batching splits, processes, merges | Handled within the single call |
| Trade-off | More round-trips; tighter control per entity | Fewer round-trips; one place to get everything right |
| Output | Identical - { home, home_assets, home_infos, asset_documents }, normalised inside the extractor | |
Because both paths normalise to the same output contract, everything after node 3 is provider-agnostic - validation, sourcing, relationship resolution and cleaning don't know or care which path ran. Switching providers is a parameter, not a rewrite.
An extraction system that returns slightly different values each run cannot be regression-tested, and a reviewer who re-runs a document set and sees different numbers stops trusting both results. So determinism is a configuration standard, not a preference.
Inference runs at temperature 0 with a fixed seed, on classification fallback as well as extraction.
Prompts are version-controlled artifacts, so a change in output can be attributed to a change in a prompt.
One schema definition per entity, referenced everywhere - not re-stated in each prompt.
Chunk classification, contextual enrichment and MAP_MERGE are switchable via environment configuration.
A 40-page inspection report can produce more relevant asset context than a single call should carry. The alternative to truncation is MAP_MERGE: the context is split into batches, each batch is extracted in parallel, and the results are merged and deduplicated as though they came from one pass.
Retrieved context is divided rather than trimmed, so no chunk is silently dropped for length.
Each batch is extracted independently, keeping latency close to a single pass.
Results are merged and collapsed by signature - the same asset seen in two batches lands once.
Two chunking strategies feed this: recursive splitting at 1000 characters with 200 characters of overlap by default, or hybrid chunking that detects document structure - headings, sections, tables, lists - preserves section hierarchy, filters boilerplate, and only falls back to recursive splitting for blocks over 1250 characters.
Across a document set the same asset appears repeatedly - in the inspection report, in the invoice that serviced it, in its own manual. Deduplication is therefore a first-class step rather than a cleanup afterthought, and each entity type gets the signature that actually identifies it.
Collapsed on the combination of type, manufacturer, model number, serial number and level. Field normalisation runs first - brand becomes manufacturer, model becomes model_number - because two spellings of the same asset won't match on an un-normalised key.
Observations are deduplicated on normalised content, which is what stops the same defect appearing once per document that mentions it. Property keys on the home entity are deduplicated too, and unit-level asset detail is filtered out of home properties rather than allowed to sit in both places.
Four extractors running in parallel means four things that can fail independently. The pipeline is built so that one failing extractor yields an empty result for its entity type and the run continues - the reviewer gets the home, the assets and the documents, and sees plainly that issues came back empty.
A failed extractor returns empty results on failure; the other three entity types are unaffected.
An asset that can't be made schema-valid is filtered out without interrupting the pipeline.
Node 8 decides whether the run is complete or goes back through extraction, with the retry count held in state.
MemorySaver persistence means a run's progress survives - recovery rather than a cold restart.
Four deterministic, non-LLM services stand between the extractors' output and the stored record. None of them use a model - they are the part of the system that has to be predictable, because their job is deciding what the language models are allowed to have produced.
Checks every entity against its JSON schema - required fields, data types, enums and value constraints, date and date-time formats, nested objects and arrays, and properties that shouldn't exist.
Uses the chunk metadata held in Qdrant to attach { doc_id, page, section } to each entity, so every value can be traced back to the document it was read from.
Validates home_asset_id on issues and related_issue_ids / related_document_ids on assets - and removes references that point at nothing.
Strips null, empty and redundant fields recursively while preserving required identifiers and metadata, reducing the payload downstream systems have to carry.
Source assignment is what turns an extracted value into an auditable one. A reviewer looking at "water heater, Rheem, installed 2016" can open the page and paragraph it came from - which is the difference between a record a carrier can operate on and a model output someone has to take on faith.
| Gate | Trigger | Outcome | Why it exists |
|---|---|---|---|
| Upload validation | Unsupported file type, or over 50MB | Rejected at the API | Nothing unparseable ever reaches the pipeline |
| Session lock | A second operation on a busy session | Blocked while busy | Concurrent runs on one session would corrupt shared state |
| Schema validation | An entity or field violates its schema | Field dropped or entity filtered | An invalid record is worse than an incomplete one |
| Reference validation | A relationship points at a missing entity | Reference removed | A dangling link is a silent data error downstream |
| Completion check | Extraction incomplete after a pass | Retry, or complete | A thin result gets another attempt rather than shipping |
| Extractor failure | One extractor errors | Empty result for that entity, run continues | Partial data with a visible gap beats no data at all |
The temptation in document extraction is to fill every field, because a full record looks like a better result. It isn't. A guessed serial number or an inferred install date is indistinguishable from a real one once it is in the database, and it will be trusted exactly as much.
So the system is built to leave fields empty, drop values that fail validation, and let the reviewer see the gap. Deterministic validation, not the model's confidence, decides what survives.
> What is the water heater's install date?
The manual gives the model and the manufacture year, and the inspection report notes the unit as "approximately 8 years old" - but no document states an installation date.
I've left install_date empty rather than deriving one, and recorded the two sources that discuss the unit's age. The asset itself is stored with its manufacturer, model number and serial number, each traced to its page.
Open the source chunks →
The proof of concept runs as a containerised stack under Docker Compose - FastAPI backend, React front end, Qdrant, PostgreSQL - with sessions isolated from each other and every model call traced from the first day rather than the first incident.
The same reading work, expressed as a deterministic pipeline with an audit trail.
Session-scoped collections holding chunk embeddings and the metadata that makes retrieval category-aware.
Why: per-session collections give multi-tenant isolation for free - one session's documents can never be retrieved into another's extraction run.
Document metadata, ingestion results and session records - the structured, queryable side of the system.
Why: the extracted record is relational by nature - a home, its assets, the issues against them - and the review UI needs to query it that way.
Automatic tracing of every LangChain operation, with model-call monitoring, token usage, performance metrics and error tracking.
Why: an audit lens and a cost lens from day one - which extractor consumed what, and where a bad field entered the record.
/api/auth - JWT login and user info/api/documents - upload, search, list, delete/api/sessions - create, list, get, update, delete/api/ingestion - extract and fetch results by ingestion, home or session ID/health - Qdrant and PostgreSQL statusWe report build facts, not projections. The figures below describe the system as scoped and implemented for the proof of concept; the before-and-after is a description of how the work changes shape, not a claimed metric. Extraction accuracy and throughput are measured on the client's own document sets once the POC runs at volume - and we would rather publish those numbers late than invent them now.
Orchestrated nodes in the extraction state machine
End-to-end processing steps, schema load to final output
Extraction agents - four specialised, one unified
Deterministic supporting services, no model involved
Linked entity types in the output record
| Before · reading documents by hand | After · the extraction pipeline |
|---|---|
| An hour or more per document set, per reader | One traced run over the whole set, unattended |
| Home facts, assets and issues separated by judgement | Every chunk classified into four categories before extraction |
| Output shape varies with the person and the document | Schema-validated entities, or an explicit gap |
| Issues recorded without the equipment they belong to | Issues linked to assets by validated references |
| The same asset entered once per document | Duplicates collapsed by signature across the set |
| No trail from a stored value back to a page | Document, page and section on every entity, traced in LangSmith |
A proof of concept earns its keep by settling arguments cheaply. This one settles three: whether category-based retrieval beats handing an extractor the whole document, which provider path performs better on real inspection reports, and how much of the record survives strict schema validation. All three answers come from the client's own documents - which is exactly why the two model paths and the feature flags exist.
The node structure, the schema layer and the provider abstraction were chosen so the system can grow - more document types, richer confidence signals, corrections that feed back - without re-architecting the core.
Classify before you retrieve, retrieve per entity, extract with narrow agents against a fixed schema, then let deterministic services validate, source and link the result. The pattern transfers to any domain where the facts live in documents written for people and have to end up in a system built for machines.
We design and ship production-grade extraction and agent systems for high-stakes work - schema-bound, validated and traceable at every step, so the output can be trusted without someone re-reading the source.