Reliable AI Begins Before the Model

The most reliable AI systems are not built on better prompts alone; they are built on knowledge that has been structured, bounded, tested, traced, and made accountable to human review.

In Knowledge as Code, I argued for the file: institutional memory that survives a tool change because humans and machines can both read it. In Knowledge Priming, I argued for the attention budget: what the agent reads first. This essay is about what happens after the files and the primer exist. Reliability is still a production problem. The fix begins before the model.

Decades of Records, and Still a Maze

In my genealogy research stack, the gold sits in scans and transcripts, not tidy tables. Witnesses hide in margins. Surnames collide. An absent name might mean truly absent, or merely unreadable. The work is not “find a document.” The work is answer a hard identity question without inventing a confident merge from a bare name.

I have shipped plenty of research prompts that looked excellent in a single session: stay on one branch, separate evidence tiers, avoid name-only merges, preserve negative results. Those instructions improved individual runs. They could not answer the production questions. Were witnesses checked in every previously processed record? Which conclusions had been reviewed? Could the same standards hold across hundreds of files? Earlier in July, an audit made the gap impossible to ignore: hundreds of person notes and role-bearing candidates still needed evidence review; nearly two thousand legacy manifest rows had no demonstrated witness check. That was a coverage and auditability problem, not a prompt-quality problem.

The same reliability pressure shows up at enterprise scale. In Building Reliable Agentic AI Systems, Sarang Sanjay Kulkarni describes PRINCE, Bayer and Thoughtworks’ Preclinical Information Center: decades of safety study reports, incomplete metadata after migrations, and multi-agent workflows that must research, validate, and draft under scientific pressure. A smarter chat box was never the finish line there either. Invented fluency is more dangerous than silence. Silence at least tells you to go look.

What Moved Outside the Model

Kulkarni names two lenses that industry discourse has finally said aloud: context engineering and harness engineering.

Context engineering shapes what each model step sees and what it must not see. Larger windows do not remove the need for discipline. Stuffing everything into one prompt makes a system harder to steer and harder to evaluate. Different stages need different context: planning, retrieval, evidence, and synthesis. That way, the agent fails fast on ambiguity instead of thrashing every source.

Harness engineering is the scaffolding around the model: orchestration, tool boundaries, state, retries, validation, observability, and human review. Governance has to be runtime behavior, not a policy PDF nobody opens. Harrison Chase’s line, as the market now repeats it, lands cleanly: the barrier between agent demos and production is usually the harness, including context management, tool control, observability, and long-running tracking. Ethan Mollick makes the same point for a wider audience: the same model behaves differently depending on the harness it runs inside.

The market already agrees that prompts alone do not make a product. The open question is what the harness is made of when the work is institutional knowledge, whether preclinical reports at Bayer or a family archive you would sign your name under.

The Reliability Check Is a Knowledge Check

Treat these five verbs as an operating check, not a slogan:

1. Structured. Knowledge is shaped for retrieval and judgment, not left as a pile of PDFs and tribal memory.
2. Bounded. Domains, tools, and sources of truth have edges. Cross-domain leakage is treated as a reliability failure.
3. Tested. Evaluation is part of the system, not a slide after launch.
4. Traced. You can see what the agent did, with what evidence, when it went wrong.
5. Accountable to human review. Someone can stop, correct, or own the commit. Review is a reliability feature, not theater that arrives after a board scare.

Inside The Knowledge Stack, that check lives at the System layer: the rituals, connectors, and review rights that keep Substrate and Semantics honest in production. Knowledge Priming decides what loads first. Negative Knowledge keeps the retirement trail. Reliability asks whether the organization built a harness around that knowledge, or only bought a model and hoped the chat window would invent governance.

This is where my argument diverges from the “harness beats the model” discourse. Most of that discourse stops at orchestration code. Useful, and incomplete. The durable harness for knowledge work includes the files, the bounds, the tests, the traces, and the named reviewers. Without those, you have a faster way to ship the average of the internet with more confidence than the average deserves.

For a platform lead, or anyone running autoresearch over a corpus they care about, the check is operational. Can you point to the authoritative source, the domain boundary, the last evaluation or audit, the trace of a bad answer, and the person who can override the agent? If any of those five are missing, you do not have a reliability problem that a new model card will fix. You have a knowledge-governance gap wearing an AI costume.

Before You Upgrade the Model Again

I learned the hard version of that check in my own stack.

The clearest turning point was incoming-scan watch and auto-file. A dry run exposed surname-only routing errors: heuristics that looked fine until they filed the wrong people into the wrong places. Outcomes improved only after we added executable confidence bounds, abstention when confidence failed, no-overwrite rules, trace output, and preview-before-apply. The model could still propose. The repository decided what was allowed to change.

Witness backfill is the mature expression of the same design. We stopped treating prompts as the operating system. A durable workflow defined completion states: pending, completed, not-applicable, backfill-needed, and blocked. Discovery tools stayed discovery-only: they could not invent identities or mutate canonical files from bare names. Batch reports preserved what was examined, what evidence existed, and why each result completed, rejected, or blocked. The first ten candidates ran as a pilot; only after human approval did those decisions become calibration rules, then institutional memory in manifests, templates, morning packets, and person-page creation, not leftovers inside one lucky conversation.

After the pilot, candidates eleven through thirty ran under those approved rules: more event appearances with provenance, no person notes from bare names, illegible evidence left explicitly blocked. The win was not simply more data. It was safer data, with uncertainty represented, decisions reproducible, and a review trail you can resume.

An earlier boundary cleanup taught the same lesson at repository scale: useful research still drifted into the wrong vault until ownership and cross-repository edges were structural. Prompts guide a run. Structures govern a system.

The portable lesson is blunt. If reliability is stuck, do not start with a bigger model. Start with the knowledge contract and the harness around it. Ask which of the five verbs is missing. Ask who owns the review gate when the agent is “done.” Ask whether your context is curated per stage or dumped into one window because the window is large. Ask what would break if the model vendor shipped a new release tomorrow. If the answer is “everything,” you do not own a harness. You rent a personality.

Teams that treat knowledge as code already have a head start: versioned files, reviewable claims, and primers that override training-data defaults. Reliability is the next layer of that stack. It is the difference between an agent that can talk about your corpus and an agent whose answers you would sign.

Kulkarni’s PRINCE write-up is not a product you need to buy. It is a mirror at enterprise scale of the same principle. Production reliability for agentic systems begins before the model: in structured knowledge, bounded tools, tested workflows, traced decisions, and humans who still hold commit rights.

Which of the five verbs is missing in the agent system you trust most today?

Madam I’m Adam

This continues the thread from Knowledge as Code and Knowledge Priming: the file and the attention budget only hold if you also build the harness that makes them reliable in production.