Shipping AI agents at 3 AM.

The best software I've ever shipped was written between midnight and sunrise, when the internet goes quiet and every prompt suddenly answers back with a little more honesty. This is a note from those hours — about building autonomous LLM systems that survive contact with real users, and the small design decisions that make AI feel less like a party trick and more like a collaborator.

1. Retrieval is a product decision, not a plumbing detail

Every team I've worked with wants “RAG.” Very few want to talk about chunking strategy, embedding drift, or the sad reality that a 400-token window with two great citations beats a 32k window stuffed with mediocre ones. Retrieval is a product surface: it decides what your model gets to think with. Treat it like UX. Prototype it like a homepage.

My default: hybrid retrieval (dense + BM25), a reranker on top, and short, opinionated chunks sized to the shape of the answer — not the shape of the source document. Ship that, then start measuring what actually gets cited. You'll be surprised how often the “obvious” document never wins.

2. Agents fail at the seams, not the steps

A single LLM call rarely breaks. What breaks is the handoff — the moment one agent turns its confident, half-hallucinated output into the input for the next one. That's where errors compound and where debugging turns into archaeology.

The fix isn't a smarter model. It's stricter contracts between steps: typed tool calls, validated schemas, retries with backoff, and a supervisor that can say “no, do that again, but shorter.” LangChain, CrewAI, custom orchestration — the framework matters less than the discipline of treating every hop as a public API.

“Prompts are code. Version them, review them, and delete the ones that stopped earning their keep.”

3. Latency is a feeling, not a number

Users don't experience p95 latency. They experience the gap between clicking and being sure something is happening. Stream tokens. Show retrieval hits as they land. Fade in the first sentence before the reasoning trace finishes. A 6-second response that starts moving at 400ms feels faster than a 2-second response that arrives all at once.

4. Evals are the only opinion that matters

Nothing has changed my engineering more than writing evals before writing prompts. Fifty hand-labeled examples with a clear rubric will tell you more than a thousand vibes. Once you can measure, you can iterate; once you can iterate, the model choice stops being a religious debate and starts being a line item.

5. The interface is half the intelligence

A well-placed empty state, a good default, a keyboard shortcut that feels inevitable — these do more for perceived intelligence than any parameter tune. The best AI products aren't the ones with the smartest model; they're the ones where the seams between human intent and machine execution have been sanded down until they disappear.

Closing note

Building with AI in 2026 is less about chasing the newest model and more about becoming a better editor — of prompts, of data, of the small moments where a product either respects the user's time or wastes it. That's the craft. Everything else is just tokens.

If any of this resonates and you're building something that needs a partner in the trenches, my inbox is open. I answer at odd hours — usually around 3 AM.