Retrieval-augmented generation has become the default answer to a reasonable concern. Organisations want AI that speaks from their own knowledge, not the open internet. So they point a retrieval pipeline at their document stores, connect a model and expect grounded answers.
The pipeline is the easy part. Embedding, indexing, retrieval and generation are solved problems with mature tooling. What isn’t solved, and what no tool can solve for you, is the question the pipeline quietly assumes has already been answered: is the corpus underneath it something an organisation would actually want to be governed by?
We call this the corpus problem, and it’s where most local RAG initiatives succeed or fail before the model ever runs.
Retrieval doesn’t find truth. It finds what you kept.
A locally harvested corpus is not a knowledge base. It’s an archaeological record. It contains the policy that was superseded and the one that superseded it. The draft alongside the final. Three versions of the same procedure, written by three teams, each internally consistent and mutually contradictory. Meeting notes that captured a decision, and later notes that quietly reversed it without referencing the first.
None of this is a failure of document management. It’s the normal sediment of an organisation doing its work. Humans navigate it effortlessly, because humans carry context the documents don’t: they know which version is current, whose authority stands behind which file and which folder nobody has trusted since 2022.
Retrieval carries none of that context. It ranks by relevance, not by authority. Ask a question that touches a contested or superseded area, and the pipeline will faithfully surface the most semantically similar passage, which may be the 2021 answer to a 2026 question. The generation step then does what it’s designed to do: it renders that passage fluently and confidently.
The system isn’t hallucinating.
It’s remembering something you forgot you still had.
The harvest is an architectural decision
Most RAG initiatives treat harvesting as ingestion: crawl the shares, chunk the files, embed everything. The implicit theory is that more coverage means better answers, and that retrieval will sort the signal from the noise.
It won’t. Retrieval amplifies whatever structure the corpus has. If the corpus encodes disagreement, the answers will encode disagreement, delivered one confident response at a time to different people asking the same question. This is the reporting coherence problem all over again, relocated from dashboards to conversation. Two executives ask the assistant the same question and receive different answers, each correctly retrieved from a different document, and now the reconciliation meeting has a new agenda item.
A designed harvest looks different. It starts with three decisions that are organisational, not technical. First, provenance: which sources are authoritative for which subjects, and which are context at best. Second, currency: how the corpus distinguishes what is current from what is historical, and who owns that distinction. Third, boundary: what is deliberately excluded, because a corpus is defined as much by what it leaves out as by what it ingests.
These decisions cannot be delegated to a chunking strategy. They are the corpus equivalent of signal architecture: the layer between raw material and meaning, and the layer most initiatives never design.
Why local corpora expose more than they retrieve
There’s a second-order effect worth naming. Pointing retrieval at your own documents is the fastest audit of organisational memory most organisations will ever run. Within weeks, the initiative surfaces every ambiguity the filing system had been politely concealing: the policies with no owner, the procedures with no dates, the folders where authority was assumed rather than assigned.
This is uncomfortable, and it’s also the value. The model didn’t create the incoherence. It found it, the same way AI initiatives generally find the architecture underneath them. Organisations that treat these findings as pipeline defects will tune retrieval parameters indefinitely. Organisations that treat them as governance findings will fix the corpus once, and every downstream capability improves with it.
The return on a governed corpus
When the corpus is treated as a governed asset rather than a harvest, the effects compound. Answer consistency improves, because retrieval draws from sources with settled authority. Trust accelerates, because users stop needing to verify every response against their own memory of which document was real. And maintenance becomes tractable, because currency is an owned process rather than a periodic re-crawl and hope.
RAG on locally harvested corpora is genuinely powerful. But the power is downstream of a decision that predates any model selection: whether the organisation is willing to say, deliberately and in writing, what it knows, what it used to know and who decides the difference.
The pipeline retrieves.
The architecture decides what’s worth retrieving.
If your AI assistant gives different answers depending on who asks, the model is rarely the problem. We can help you design the corpus underneath it.
Begin a Conversation