Processes · Knowledge-driven development, part 5 of 5
Where it breaks
Knowledge-driven development, part 5: a wrong map is worse than a blank one
Seeded September 2026 Written October 2026
The thing about knowledge-driven development that irritates me most is the marks that don't go away. A ⚠️ or a ❌ is meant to dissipate fast. Mine pile up, and the only reason is that I don't have the token budget to clean my desk.
The list below is short partly because the method is young. I've been running it for a couple of months, not years. It isn't battle-tested yet, and the problems I haven't met yet aren't on it.
1. A wrong mark is worse than no mark
A ✅ next to something nobody ran is a lie that the next reader builds on. That would kill the method, because ✅ is the mark people trust.
A blank map tells you to be careful. A wrong map tells you to walk off the cliff with confidence.
A false ✅ has never bitten me. I don't let the documents and the code drift that far apart, and I'm a stickler for unit tests and branch coverage. The first defense is that ✅ is expensive to write. No build can check a ✅ on a paragraph about how an operator handles a jammed door. Nobody's coverage tool has an opinion about doors.
An agent does, if you ask it. Checking the marks is part of every review: the agent that checks the claims reads the code behind each mark that moved. A wrong mark only gets serious when nobody updates the document, and the re-review is there to make sure that never happens.
So I don't take an old document seriously until it's been checked. When I want to know a partially implemented feature in detail, and the page describing it hasn't been touched in a while, I have an agent give it a once-over against the code first. Then I read it.
2. Marks go stale
Marks don't rot on their own. They go stale the way any document does: the code moves and the page doesn't. With several agents working on parallel branches, the author of a change often doesn't know which pages describe the thing they touched. A stale ⚠️ against a capability that now exists sends the next agent off to build it twice.
There's no need for a mechanical fix. Ask an agent to review the page, on request or on a schedule. Mechanical checks of that kind break anyway.
3. One kind of document isn't enough
"Upgrade to the next framework version" belongs to none of the capability documents. Neither does "the build is slow" or "the test runner leaks file handles". Work that isn't about a capability has no natural place in them.
That's why knowledge-driven development needs more than one type of document: capabilities, features, requirements, architecture, the build, operations, and so forth. The framework upgrade becomes a page in the architecture documents, describing the system on the new version, marked ⚠️ until it's true. The slow build becomes a page describing the build as it should be. Same marks, same greps, same rules.
Adding a type is cheap. An agent reverse-engineers the current state of that area into a knowledge document, and a human reviews it. The real decision is which types are worth having in your system.
4. It doesn't schedule anything
A document says what's missing. It never says when it'll be done or who owns it, and it isn't supposed to.
In an ideal world, I wouldn't schedule anything. I'd throw agent tokens at every ⚠️, and the problems would disappear. In the real world, I schedule them against a meagre token budget. When I do, I create a task in Asana, and the task points at the knowledge document. The task says when. The document says what, and whether it's true yet.
5. Gates, or an agent
A convention maintained by attention decays at the rate attention decays. A few rules earn a build failure. The rule against TODO was written down, and needed one.
When I wrote the specification for the gates, there were seven of them: the coverage cross-check, the test-location gate, the API contract scan, the vocabulary gate, the deferred-work gate, the orphan-⛔ check and the graduation check. One of them existed. The rest were described in the present tense and marked ⚠️, which is to say the enforcement spec was written using the method it enforces.
The missing ones aren't serious. Where a rule has no gate, I ask an agent to check the documents against it, or to do the work the rule was protecting, and it does. If I have the tokens.
A spec nobody maintains is a lie with a schema. A knowledge document nobody maintains is the same lie with a green tick on it. The difference is that you can ask an agent to check this one before you believe it.