--- title: "Skills are documents too" date: 2026-05-08 author: "Daniel Henneberger" description: "LLM knowledge is jagged. Treating skills as composable, validated documents is how we make the edges less surprising." tags: ["skills", "docs", "agents"] cover: "/blog/skills-are-documents-too.svg" --- LLM knowledge is jagged. That is the thing people underestimate after their first good demo. A model can explain a distributed consensus protocol, then miss some boring detail about how your repo runs migrations. It can write a clean TypeScript type, then forget the exact flag your team uses in CI. The failures are rarely evenly distributed. They show up as weird little cliffs. Skills are how we sand those cliffs down. ## Skills are documents A skill is not magic model dust. It is a document with a job. It tells the model how your team does something: how to review an API change, how to write a migration note, how to summarize an incident, how to generate architecture docs without inventing abstractions that are not in the code. That matters because documents are inspectable. They can be reviewed, versioned, tested, improved, and deleted when they stop being true. A skill should have the same operational shape as the rest of your engineering knowledge. If a skill only works because the author had a vibe in their head, it is not a skill yet. It is folklore with a filename. ## The jaggedness problem Frontier models are strong in aggregate and uneven in detail. One model may be excellent at repo exploration but sloppy with product voice. Another may be careful with factual extraction but too timid when it needs to propose a patch. A third may know a library deeply because it saw a lot of public examples, then know almost nothing about the private conventions that decide whether your team accepts the PR. So we evaluate skills against all frontier models. Not because every customer should care which model wins a benchmark this week. They should not. The point is to find the rough edges in the skill itself. If a workflow only succeeds with one model, the skill is probably carrying hidden assumptions. If several strong models fail in different ways, those failures tell us where the document is under-specified. The goal is not to make the model omniscient. The goal is to make the task less dependent on luck. ## Smaller skills compose better Large skills are tempting because they feel complete. In practice, they rot faster and transfer worse. We break skills apart so they compose around best practices: - One skill can describe the review posture. - One can describe the fact base for a framework, product area, or API. - One can describe the output format. - One can describe the validation loop. That separation is not bureaucracy. It is how the model can bring in only the context it needs. A migration-writing task should not load your entire incident-response playbook. A docs-generation task should not inherit a code-review rubric unless it is actually reviewing code. Small skills also make disagreement visible. If the factual skill is wrong, fix the facts. If the reasoning workflow is weak, fix the workflow. Do not make one giant document where every correction risks changing the model's behavior somewhere else. ## Chain-of-thought skills and fact skills are different Some skills teach a procedure: inspect the repo first, identify source-of-truth files, check tests, produce a bounded patch, cite evidence. These are workflow skills. They are about how to think through the task. Other skills carry facts: the current architecture, the public API contract, the house terminology, the exact deployment topology, the list of supported integrations. These are fact skills. They are about what is true right now. We keep those separate. The reason is simple: facts change more often than good reasoning patterns. If a deployment detail changes, you should not have to touch the whole investigative workflow. If the workflow improves, you should not risk editing product facts by accident. This also gives the LLM a cleaner choice. It can pull in a procedural skill when it needs a disciplined loop, and a fact skill when it needs authoritative local knowledge. Mixing those together makes the prompt look rich, but it makes the system harder to validate. ## Generated architecture docs are skills-adjacent Architecture docs have the same shape. They are documents generated from code, but the hard part is not generation. The hard part is validation. Did the doc find the real boundary between services? Did it notice the background worker? Did it miss the one table that controls the whole workflow? Did it infer a queue where the code actually makes a synchronous call? Mari checks the facts in a generated architecture doc against your code, then asks what the document missed. That second pass matters. A doc can be fluent and still be wrong in exactly the place a new engineer will trust it. We would rather be pedantic than smooth. Pedantic accuracy is the product: source-backed facts, named uncertainty, and a bias toward saying less when the code does not support more. The same principle applies to skills. A skill should not merely sound like a best practice. It should survive contact with real tasks, real repos, and multiple frontier models with different failure modes. ## The document is the interface The most useful thing about skills is not that they make the model smarter in some abstract sense. It is that they turn model behavior into something teams can edit. When the agent does the wrong thing, you can ask a concrete question: which document failed? - Was the fact outdated? - Was the workflow underspecified? - Was the task too broad? - Did the validation step fail to catch the miss? That is a better debugging surface than "the model hallucinated." It gives the team a place to put the fix. Skills are documents too. Treat them that way and they become part of the engineering system: composed carefully, evaluated repeatedly, and kept honest by the code they are supposed to describe.