Enterprise AI agent architecture: keep knowledge in git

Meta published an architecture on its engineering blog for what it calls an organisational second brain: an agent system that holds institutional knowledge in more than 200 structured files under strict taxonomies, and improves by compiling expert corrections into verified text edits rather than by retraining anything.

That last part is the whole idea. Most enterprise AI agent architecture discussions are about which model, which framework, which vector store. This one is about where the knowledge lives, and it answers with version control.

The four layers

As described by InfoQ, the system has four parts:

• A knowledge system. 200+ structured files in strict taxonomies — position files, vocabulary files, routing indexes, gateway files.
• A reasoning pipeline. Composable "recipes" that specify what to examine, which knowledge to load, the decision procedure, and what counts as complete.
• An evaluation framework. Automated benchmarks for assessing performance.
• A self-improvement loop. Expert feedback compiled into verified, regression-tested updates, without model retraining.

The reported outcome is individual assessment time falling from days to minutes, with experts judging results useful almost universally, and zero regressions across improvement cycles.

Why version control is the load-bearing choice

Consider what happens in a typical agent deployment when an expert says the agent got something wrong.

Usually the fix is a prompt edit. Somebody appends a sentence to a system prompt, or adds a few-shot example, or nudges a retrieval threshold. It works. Nobody records why. Six months later that prompt is a 4,000-word document of accumulated corrections, nobody can say which sentence is load-bearing, and removing any of it feels dangerous — so it only grows.

Meta's answer, in their framing: expert corrections are diagnosed to their root cause, compiled into minimal verified edits, and evaluated against replay and regression tests.

Unpack that and there are four separate disciplines, each of which is the thing most teams skip:

Diagnosed to root cause. Not "the agent said something wrong, add a rule". Why did it? Missing knowledge, wrong routing, an ambiguous vocabulary term? The fix depends on the answer and lands in a different file.

Compiled into minimal edits. The smallest change that addresses the cause, rather than a defensive paragraph that covers the symptom and three imagined neighbours.

Verified. The edit demonstrably fixes the case that prompted it.

Regression tested. And it breaks nothing that previously worked — which is the property that lets the knowledge base grow past the size one person can hold in their head.

What you get for free from text files

Putting knowledge in version control rather than in weights or an opaque store buys properties that are ordinary in software and rare in AI systems.

Changes are diffable, so you can see exactly what an improvement changed. They are attributable, so you know which expert made which call. They are reviewable before they take effect. They are revertable when they turn out to be wrong. And they are greppable, so "what does the system believe about X" is a question with an answer rather than an experiment.

None of that is available when the correction went into a fine-tune, and only partially available when it went into an embedding store.

What a recipe is doing

The reasoning layer is the part most likely to be skipped by anyone copying this, and it is the part that makes the knowledge layer usable.

A recipe specifies four things: what to examine, which knowledge to load, the decision procedure, and what counts as complete. Read that as a refusal to put the whole knowledge base in the context window and hope. With 200+ files, loading everything is neither affordable nor effective — the relevant passage competes with two hundred irrelevant ones.

Naming which knowledge a given task loads does two useful things. It makes the retrieval decision explicit and reviewable rather than a similarity score, and it makes failures diagnosable: when the answer is wrong you can ask whether the right files were loaded before you ask whether the reasoning was sound. Those are different bugs with different fixes, and a system that cannot distinguish them tends to accumulate prompt text aimed at whichever one somebody guessed.

The completion criterion matters too. An agent that does not know when it is finished either stops early or keeps going, and both look like quality problems rather than the specification gap they are.

The part worth arguing with

This is Meta's account of Meta's system, and two caveats are worth holding.

"Zero regressions across improvement cycles" is only as strong as the regression suite. A suite built by the same team that writes the edits will tend to encode the same assumptions, and a regression nobody thought to test for is indistinguishable from no regression. The claim is really "no regressions we test for", which is still good and is not the same sentence.

"Days to minutes" is a comparison against a manual baseline we cannot see. The honest reading is that a slow, expert-driven, mostly-reading task got much faster, which is the category where these systems genuinely do work — not evidence that it generalises to tasks with different shapes.

Neither undermines the architecture. Both are reasons to copy the structure rather than the numbers.

What transfers to a company that is not Meta

200+ taxonomy files with a routing layer is not a starting point for anybody. The transferable parts are smaller.

Put the knowledge somewhere a non-engineer can edit and an engineer can review. Markdown in the repository is usually enough. The requirement is that a domain expert can change what the agent believes without a deployment, and that somebody sees the change before it ships.

Separate knowledge from procedure. Meta's split between knowledge files and recipes matters because they change at different rates and by different people. What the company knows changes weekly; how an assessment is conducted changes rarely. Mixing them into one prompt means every factual correction risks the procedure.

Build the regression suite before you need it. Ten to twenty real cases with known-good outcomes, run on every knowledge change. This is a day of work at the start and it is the difference between a knowledge base that can grow and one that quietly rots.

Make the improvement loop somebody's job. A correction that arrives in Slack and is fixed by editing a prompt has not improved the system; it has patched an instance. The loop only compounds if there is a route from feedback to a reviewed, tested edit.

One more, and it is the one clients push back on: write down what the agent is not allowed to decide. Meta's system keeps human checkpoints and expert escalation specifically so a wrong decision does not propagate. That boundary is easier to define at the start, when nobody trusts the system yet, than a year in when it has been right often enough that removing the checkpoint feels like reasonable efficiency.

When this is the wrong shape

This architecture suits work that is knowledge-heavy, judgement-based, repeated, and currently done by experts reading things. Assessments, reviews, triage, applying policy to cases.

It is overkill for an agent that answers questions from documentation you already have — that is retrieval, and adding a taxonomy layer to it is ceremony. It is also the wrong shape where the knowledge genuinely changes faster than a review cycle can keep up, because the review step becomes the bottleneck rather than the safeguard.

And it does not help if the underlying task has no experts to learn from. The self-improvement loop is compiled expert feedback. No experts, no loop.

The recommendation this changes

When we build AI systems for clients, the question that decides most of the architecture is where corrections go. Teams almost always answer "the prompt", because it is fastest on day one and it is the default the tooling encourages.

The version we would now argue for on any system meant to last: knowledge in the repository, procedures separate from facts, a small regression suite from the first week, and one named person who owns turning feedback into reviewed edits. It is slower for a month and it is the only version that is still improving in year two.

If you are standing up an agent that has to encode how your organisation actually makes a decision — rather than answer questions about documents — that architecture conversation is worth having early, because retrofitting version-controlled knowledge onto a system built around one large prompt is close to a rewrite. It is the same lesson as making a codebase AI-ready: the structure decides what you can do later.