Retrieval-Augmented Generation solved a real problem. It let language models answer questions about documents they were never trained on, without retraining anything. But after running RAG in production for long enough, a different problem emerges: the system retrieves the same paragraph, reasons over it, produces a good answer, and then throws all that reasoning away. Tomorrow, a related question arrives, and the work starts from scratch.
This is not a retrieval problem. Retrieval is doing exactly what it was designed to do. It is an accumulation problem. Standard RAG has nowhere for understanding to build up. No amount of semantic caching or re-ranking fixes it, because those optimize the lookup, not the memory.
The Hybrid Knowledge Architecture
The proposed architecture adds a persistent knowledge layer on top of the evidence layer. The evidence layer remains a retrieval index of chunked, embedded documents. The knowledge layer is a maintained, structured, human-readable model of the domain, stored as typed objects in Cosmos DB and exported to Markdown for auditability. An orchestrator decides whether a question needs raw evidence, compiled knowledge, or both.
The critical rule is that the knowledge layer never replaces the evidence layer. A generated page is a derived interpretation, not evidence. The moment it pretends to be evidence, the system becomes dangerous.
The Object Model That Matters
The design defines several object types, but three carry the architecture:
Decision objects store a rule, its scope, its effective date, an accountable owner, and its rationale, with a pointer back to the source. In the demonstration corpus, an underwriting update says roof inspections trigger above 15 years in one wind zone. The rationale exists only in an email thread warning that people will read the number without the qualifier. A retrieval system would never rank that email highly. The knowledge layer preserves it deliberately.
Contradiction objects capture conflicts between current documents. Two teams write two guidelines that disagree. A RAG system retrieves one chunk, or both, and asks the model to reconcile them. The model picks a side, usually the newer document, which is often wrong. The knowledge layer detects the conflict, stores both statements verbatim with their effective dates, names an owner, and refuses to settle the dispute.
Open question objects track questions that cannot be answered yet, often because they are blocked by an unresolved contradiction. An unanswered question is safe. A quietly wrong answer ends up in a complaint file.
Six Traps That Retrieval Falls Into
The author built a 21-document synthetic insurance corpus specifically to test where retrieval fails. A pure RAG system falls into every trap, and falls fluently.
- Scoped rules: A 15-year roof threshold applies only to new business in zone H3. Retrieval returns "15 years" without the qualifier.
- Contradictions: One manual says trace-and-access is standard. Another says it is a paid endorsement. Retrieval picks one.
- Entity resolution: The same concept appears as "ACV," "actual cash value," and "depreciated value." Without resolution, the wiki forks into disagreeing pages.
- Temporal validity: A rule took effect March 1, 2026. A claim occurred February 20. Semantic similarity cannot encode this.
- Rationale recovery: The reasoning lives in an email, the rule in a guideline, and the connection between them lives nowhere.
- Multi-hop traversal: Answering why a claim was triaged at Level 1 requires claim notes, then the triage guideline, then the water-damage concept, then the policy clause. Similarity search ranks; typed relationships traverse.
Governance and Cost
The moment a system writes persistent knowledge, it crosses from retrieval application to system of record. The model never writes directly to the store. It proposes a patch. The application validates it, and a human approves consequential changes. Resolving a contradiction is never automatic.
The write amplification is real. A hybrid pipeline summarizes, extracts concepts, resolves entities, generates relationship patches, validates, and regenerates Markdown. But measured on the deployed Azure stack (Foundry, AI Search, Cosmos DB, Container Apps), ingesting all 21 documents and running every walkthrough cost roughly 0.05 per day. At this corpus size the compilation cost is negligible. The economics only become interesting at scale, where the break-even point against simple RAG sits at roughly 117 questions for a 50-document corpus.
The repository, including FastAPI application, Bicep infrastructure templates, synthetic documents, and an Obsidian vault export, is available on GitHub under the MIT license.