The knowledge-base problem at 10,000 pages

There is a moment in most large organisations when the knowledge base stops being a single thing anyone can hold in their head. It usually happens somewhere between four and six thousand pages. By ten thousand, the team is no longer working with the corpus — they are working with their memory of the corpus, and a small number of trusted experts’ memories of it. The information is intact. The ability to traverse it under time pressure is not.

This is the knowledge-base problem. It is not a problem of missing material. It is a problem of access to the material you already have, at the moment you need it.

How the bottleneck moves

In the early phase of any large knowledge effort the bottleneck is collecting the material. Reports get commissioned, technical experts give their views, partners contribute, and the corpus builds up. Most of the discipline goes into that collection: making sure the right people are engaged, the right scope is set, the right questions are answered.

Somewhere along the way the bottleneck migrates. The material is there, but the team’s ability to answer specific questions from it — quickly, accurately, in a form a board or a regulator or a client will accept — degrades faster than anyone expects. By the time a deadline lands, the work is no longer about producing new material. It is about traversing existing material under time pressure that the original collection process never accounted for.

What experts actually do at this point

A senior specialist at week three of a high-stakes work cycle is doing four things that look like one thing. They are recalling which document a piece of evidence sits in. They are checking that the document version they’re looking at is the current one. They are reconstructing how that evidence connects to the questions actually being asked. And they are translating all of it into language a non-specialist decision-maker can act on.

Each of those four tasks is the kind of thing AI can support, but only if the system is built for the work. A model that can summarise a single document doesn’t help — that wasn’t the bottleneck. The bottleneck is connecting evidence across the corpus while being certain you are working from the right versions and the right context.

The information isn’t the problem. The ability to find what you already have, in the form you need it, at the moment you need it — that is the problem.

Why the right system pays back fast

When the system is right, three things change. Specialist time shifts from retrieval to judgement. Response cycles compress from days to hours without sacrificing accuracy. And — quietly, but importantly — the team’s confidence in its own corpus goes up, because every claim is one click from its source.

That last point is the one we underestimate. When the team can stop carrying the corpus in their heads, they stop being afraid of it. The work gets sharper. The reporting up gets cleaner. The reporting out — to regulators, clients, communities, media — gets more confident, because nothing is more than a citation away from being verified.

What it isn’t

It isn’t about replacing specialists, or "automating" judgement. The hard parts of serious knowledge work — the strategy, the politics, the reputational instincts — are exactly where senior people should be spending their time. The point of the system is that they can spend that time on those parts, instead of spending it reconstructing where in the corpus a particular piece of evidence sits.

At ten thousand pages, the question isn’t whether to bring AI into the work. It is whether to bring in AI that was built for the work, or AI that was built for something else and reskinned for it. That is the actual decision in front of most teams now.