We spent three weeks blaming the prompt. It was the retriever the whole time.
The failure mode looked like a prompting problem — vague answers, confident hallucinations, inconsistent tone. It wasn't. The retriever was surfacing the wrong chunks, and no amount of prompt engineering was going to fix retrieval quality. The fix was boring: better chunking, per-document retrieval settings, and a way to see exactly what got retrieved for a given answer. Debuggability beats cleverness every time.
· 4 min
7 things I do differently after 4 years of shipping.
Ship the ugly version first. Row-level security from day one, not after the first incident. Treat derived data stores as caches, never sources of truth. Write the onboarding doc before you need it, not after the second hire. Measure what a feature costs in support tickets, not just in build time. Trust the framework's conventions until you have a specific reason not to. And the biggest one: a system that's observable is worth more than a system that's clever.