RAG that stays current: knowledge that ships with the feature

Most retrieval systems do not fail at retrieval. They fail because the paragraph they found is an accurate description of software that stopped existing in July. Keeping RAG up to date is a scheduling problem before it is an engineering one, and the RAG best practices worth arguing about are mostly about when the knowledge gets written, not how it gets chunked.

My answer is a line in a deploy checklist. Every admin feature I ship writes, in the same branch, the chunks that describe it, then runs the embed step. The staff assistant can explain the dashboard as it exists today because the two are the same commit.

Why does a RAG knowledge base go stale?

Because nothing in the normal build process forces the knowledge to change when the code changes. Writing the knowledge base is a launch task. Features ship every week after that, and no pull request template asks "does this make a document wrong."

Retrieval cannot help you here. A stale chunk embeds as confidently as a fresh one, scores as high, and reads to the model with the same authority. A missing document produces "I don't know," which is annoying and safe. A stale document produces a wrong answer in the tone of a right one.

I found the sharpest version of this in my own repo in September. A prompt file told two agents that a particular function dropped three fields before handing data to the API. It had not dropped them since the day the cards were wired, five days earlier: another file mapped all three, and the view had been drawing them the whole time. Three consecutive rewrites of that prompt repeated the claim. Nobody caught it because the claim was prose, and prose does not fail.

The rule: knowledge ships in the same cycle as the feature

The standing order lives in this project's deploy checklist as item seven:

Admin/dashboard builds always feed staff knowledge: whenever an admin feature is added or meaningfully changed, ship an idempotent migration adding/refreshing audience='admin' chat_knowledge chunks about it, run it locally, and run 2026-07-06-semantic-rag.php to embed, in the same cycle as the build. Staff mode should always be able to explain the dashboard as it exists.

I wrote that on 21 July 2026, in the middle of building a Marketing tab, after noticing that the assistant sitting inside my own dashboard was describing a version of it from three weeks earlier. The unit of work is not "the feature" plus "the docs, later." It is one branch: the feature merges, a migration writes the chunks, the embed step fills the vectors. It has held. Roughly one migration in five in this repo exists only to write or refresh knowledge.

What does a maintainable knowledge chunk look like?

A row, not a document. Title, body, tags, a topic, and an audience, written by whoever just built the thing, while they still remember which button does what. Here is the whole write path from the migration that first applied the rule:

$ins = $pdo->prepare("INSERT INTO chat_knowledge (audience, topic, title, content, tags, updated_at, embedding)
    VALUES ('admin','marketing',?,?,?,?,NULL)");
$del = $pdo->prepare('DELETE FROM chat_knowledge WHERE title = ?');
foreach ($chunks as [$t, $c, $tags]) {
    $del->execute([$t]);
    $ins->execute([$t, $c, $tags, $now]);
}

Three decisions are doing all the work in those six lines.

Delete, then insert, keyed on the title. That makes the migration idempotent: run it on a laptop, run it again after editing a sentence, run it on the server, and the result is one row per title every time. Reruns are how the same knowledge reaches every environment, so a knowledge migration that is not safe to rerun is not finished.

The embedding column is NULL on purpose. Writing a chunk is a claim that its vector is stale. The embed step later selects exactly the rows WHERE embedding IS NULL OR embedding = ''. That is the entire incremental-update design, and it needs no timestamps, no change feed, no queue.

Audience is a column, not an instruction. Retrieval builds its filter as audience IN ('public') for a visitor and audience IN ('public','admin') for a signed-in staff session. The wall between what the public assistant knows and what the staff assistant knows is a SQL predicate, not a line of prompt asking the model to be discreet.

The one thing delete-by-title does not solve is the chunk you rename or retire. Microsoft's indexer documentation states the general shape of that problem plainly, and it is worth reading even if you never touch Azure:

Although change detection is a given, deletion detection isn't. An indexer doesn't track object deletion in data sources. To avoid having orphan search documents, you can implement a "soft delete" strategy.

I hit that orphan in my own pricing knowledge: an early chunk describing a posture of never quoting prices, which stopped being true the day I published a pricing page, still carrying an email address I had retired. The migration that fixed it ends with a sweep, rewriting that chunk by title and running a SQL REPLACE across every chunk still holding the old address, embedding = NULL on each. Orphan cleanup belongs in the migration, not on a list for later.

What does it cost to re-embed everything?

Less than it costs to write clever incremental logic, at least at my size. Measured today rather than remembered: the table holds 283 chunks, 137 public and 146 admin only, totaling 254,527 characters, roughly 64,000 tokens on a four-characters-per-token estimate.

The embed step batches twelve chunks per request. I ran one real batch against the live endpoint while writing this: HTTP 200, twelve vectors of 2,048 dimensions each, 1,199 prompt tokens, 0.69 seconds. Rebuilding the whole table from nothing is 24 of those requests with a 150 millisecond pause between them, so call it twenty seconds. At OpenAI's published list price of $0.02 per million tokens for text-embedding-3-small, the entire base would cost about a tenth of a cent to rebuild. My own embeddings run on NVIDIA's hosted endpoint on a free tier; the OpenAI figure is there because it is published and comparable.

Which raises a fair question, since Anthropic's contextual retrieval writeup says that if your knowledge base "is smaller than 200,000 tokens (about 500 pages of material), you can just include the entire knowledge base in the prompt." Mine is a third of that. So why retrieve at all?

Three reasons, and only one is money. Prompt cost and latency are paid per message, not per deploy. The public assistant runs on free-tier providers whose context windows are far smaller than the frontier models that guidance assumes. And the audience wall is enforceable in SQL, which beats any amount of prompt text telling a model not to mention the admin panel to a stranger.

One thing does force a full rebuild: a model change. On 25 August 2026 the embedding model I had been using started answering HTTP 410. Vectors from two different models are not comparable, so the fix is not "embed the new rows with the new model," it is "throw away every vector and start again." The step now probes its configured model, walks a fallback ladder, writes the survivor back into settings, and records which model built the index:

$ladder = [$model, 'nvidia/nemotron-3-embed-1b', 'snowflake/arctic-embed-l', 'nvidia/embed-qa-4'];
/* ... probe each until one answers 200, persist it ... */
if ($indexed !== $model) {
    foreach (['chat_knowledge', 'site_index'] as $t) $pdo->exec("UPDATE $t SET embedding = NULL");
}

Setting every embedding to NULL is the cheapest disaster plan I know, because NULL is already the instruction the incremental path understands. The rebuild is not a special mode, it is the ordinary path given more work.

Does chunk size still matter when a person writes the chunks?

Yes, and on current evidence it matters about as much as which embedding model you buy. The largest recent comparison I can find, published on arXiv on 7 March 2026, benchmarked 36 segmentation methods across six knowledge domains and five embedding models. Paragraph Group Chunking led at a mean nDCG@5 near 0.459; fixed-size character chunking, the default in most tutorials, came in below 0.244 with Precision@1 of two to three percent. The authors' summary of the interaction is the line I keep returning to: larger embedding models "yield higher absolute scores but remain sensitive to suboptimal segmentation."

That research is about splitting documents that already exist. Hand-written chunks skip the splitter, which is the quiet advantage of knowledge as rows: each is authored as one self-contained answer carrying its own title and context. That is close to what Anthropic automates with contextual retrieval, prepending a generated preamble to each chunk before embedding. Their result, from 19 September 2024: contextual embeddings alone cut top-20 retrieval failures by 35% (5.7% to 3.7%), contextual BM25 alongside them took it to 49%, and a reranking pass reached 67% (5.7% to 1.9%).

The keyword half of that finding is why my retrieval is hybrid. Full-text and vector search fail in different directions, so both run and the scores are blended.

Two chunks reached along the graph edges are appended to the five the blend chose, because some questions are answered by a connected chunk that looks nothing like the query. I wrote about the shape of those in the questions similarity search can't answer, and about when edges earn their keep in graph RAG against vector RAG.

Writing this turned up a flaw in my own pipeline. The embed call truncates each input at 2,000 characters, a limit I set when the previous model's window was short; the model I run now lists a maximum sequence length of 32,768 tokens. Eleven of my 283 chunks run past 2,000 characters, the longest at 2,807, so their tails are findable by keyword and invisible to the vector. Raising the cap is a one-line change. Splitting those eleven into shorter self-contained chunks is the better one, and it goes into the next knowledge migration.

How do two different models share one knowledge base?

By not making it a database. The second knowledge base I maintain is a folder of markdown in a private git repo, read and written by Claude Code and by a Grok-based bot organization working the same files, one room per account. Its rules file opens by calling it "the studio's shared memory. Both bots read it before working and write to it after. Git is the sync: pull first, push after." Five files, five jobs:

The house rules are short enough to hand to a new agent whole: one fact, one place. Absolute dates, never "yesterday." Sign what you write. No secrets. Past about 200 lines, split the topic. And the one that keeps it honest: if it is not verified, it goes in questions.md, not into a topic as a fact.

Plain files buy things a vector store cannot. Any runtime reads them with no client library and no embedding call. A human reviews a knowledge change as a diff, in the same pull request as the code. "When did this become true" is a git question with an exact answer. And a merge conflict in that folder is a real signal that two agents believe different things.

Freshness here is a ritual rather than a pipeline: pull, append one dated line to the month's log, update the index if a topic was added, commit the knowledge with the work it belongs to. The rules file says why better than I would: an unpushed note is a note nobody else has. The same discipline runs on the operational side, where every desk keeps a status file of its own.

What happens when the notes and the code disagree?

The notes win, silently, for as long as nobody re-reads the code. That is the failure from the top of this post, and the fix is the piece I would keep if I had to throw out the rest: the most useful document in that repo is generated, not written.

A CLI script walks the source and writes a manifest, stating no fact it did not just read off disk. It traces every filed field from the data files through the API mapping to the view that draws it, and asserts things a human would otherwise claim from memory. Run with --check it writes nothing and exits non-zero if an assertion has stopped being true. The header of the file it produces warns its own readers:

Generated by php monitor/manifest.php. Do not hand-edit. Every line below was read off the source at generation time, which is the whole point: prose written from memory is what put a false LOOK gap in the scan index for three rewrites running.

Today it reports six assertions passing, none failing. The rule I took from building it: any knowledge claim a script can verify should be produced by that script. Hand-written knowledge is for judgment, intent, and decisions. Generated knowledge is for structure, wiring, and counts, the things that drift on their own while everyone is busy, and it should be the part that can fail out loud. That split is load-bearing in an agent harness.

The part I have not solved

The eleven long chunks go into the next migration, and that one is easy. The harder gap: a chunk can be perfectly fresh and still wrong, because the person who wrote it misunderstood the feature on the day it shipped. Idempotent migrations catch drift. They do not catch a mistake made confidently at the start, and neither does a manifest, which only proves the code agrees with itself.

The only test I trust for that is opening the assistant in staff mode, asking it questions I already know the answer to, and reading the replies with an unfriendly eye. I do that perhaps once a month, when something reminds me. It should be a scoreboard with a number on it, run on a schedule, the way I run other things as loops: a fixed set of questions, a graded answer key, and a failing score the week the knowledge falls behind the software again.