Key points
- Thariq Shihipar says if model development froze, the field would still uncover capability overhang for another six to twelve months.
- Shihipar says Anthropic rewrites much of the Claude Code harness constantly, and that evals are an art most startups should skip.
- Productivity matters only when it converts to revenue, he says, and buy-in and tenacity, not build cost, remain the constraint.
Anthropic Engineer Says AI Models Outrun Their Tools
Anthropic's Claude Code team treats the model as the fast-moving part and the harness as the part that must keep up. Thariq Shihipar calls the gap capability overhang, and says models are smarter than any interface lets them express. That gap explains why the team rewrites its harness constantly and why productivity gains only matter when they reach revenue. Founders should build around the labs, not compete with them.
Takeaways
- Capability overhang defines the current AI moment. Shihipar defines it as the gap between what models can do and what their harness lets them express. If model development froze, he says, the field would spend six to twelve more months discovering the overhang.
- Models improve in spiky, unintuitive ways. The 2024 consensus held that coding would be solved by 100-million-token context windows that fit an entire codebase. Instead, models learned to build their own context, and improvements arrive unpredictably.
- Evals are an art, not a science. Anthropic rarely uses evals for new feature development, and Shihipar says many startups should skip them. Building them requires high-skill people doing boring work, and each complaint needs its own eval.
- Harnesses must be rewritten, not inherited. Anthropic rewrites much of the Claude Code harness constantly to stay on the frontier. Shihipar contrasts that with startups stuck on a v1 memory system they cannot replace, and calls RAG for context search an anti-pattern in favor of grep.
- Productivity only counts when it becomes revenue. Shihipar talks to customers who could double their pull requests without changing revenue. Org design and buy-in remain the constraint, and tenacity to make something has not decreased. Build cost is not the bottleneck.
- Educating humans matters more as models improve. Shihipar introduced ask user questions, which lets the agent stop and clarify what the user wants during planning. He argues that as models get smarter, teaching humans how to work with them matters more than polishing agent UX.
Inside the Claude Code workflow
- The workflows feature builds a custom harness on the fly for tasks like deep research.
- Much of the team has stopped using Plan Mode because the model now plans correctly on its own.
- Shihipar works in the terminal and runs at most two or three agents at once.
- Some colleagues use GitHub issues as state. Anthropic is not monolithic about self-recursive harnesses.
How to build alongside the model labs
- Make locked data sources legible to agents.