We Let AI Review Every Pull Request for Three Months. Here's What It Actually Caught.
AI code review tools are everywhere now. After rolling one out on a real project, I learned more about our team than about the tool itself.
Lab notes
14 notes
#ai.
What I Actually Learned Using AI as a Pair Programming Partner
Practical insights from months of using AI-assisted coding in real client projects — what works, what doesn't, and how it changed my workflow.
AI Tooling Won't Fix Your Broken Engineering Culture
The latest DORA report confirms what I've seen on consulting engagements — AI tools amplify existing conditions. If your processes are solid, AI accelerates you. If they're not, you just create technical debt faster.
I Audited a Codebase That Was Vibe-Coded in Two Weeks
A startup founder built their MVP almost entirely with AI coding agents. It worked. Then they hired a team, and within two months nobody could ship anything. I got called in to figure out why.
More PRs, Slower Delivery: The AI Agent Throughput Trap
A client's team tripled their PR output with coding agents. Six weeks later, their cycle time had doubled and nobody could explain why.
Reviewing AI Code Is the New Bottleneck
Last quarter I tracked my time across three consulting engagements. I spent 11 hours a week reviewing AI-generated code and under 8 writing my own. The ratio surprised me, and it changed how I think about developer productivity.
The AI Coding Bill Nobody Budgeted For
GitHub Copilot moved to usage-based billing and a client's engineering budget lost all predictability overnight. Developer tools now bill like cloud infrastructure, and most teams aren't ready for it.
The Codebase That Slowly Stopped Agreeing With Itself
A client's AI-assisted codebase passed every lint check and had solid test coverage. But under the surface, modules had drifted into contradictory patterns — three retry strategies, four error-handling philosophies, and a data validation layer that depended on which AI session wrote the code.
The Coding Agent That Could Push to Main
A client's AI coding agent had contents:write on every repo in the org. When I asked who approved that, the answer was "the setup wizard." The defaults are the danger zone.
The MCP Servers Nobody Audited
A client connected 11 MCP servers to their development environment in three months. When I reviewed the OAuth scopes, two of them had write access to production infrastructure. Nobody had checked.
The Team That Had More AI Agents Than Engineers
A client's platform had 23 AI agents built by a team of 8. Nobody could tell me what half of them did. Agent sprawl is the new microservices sprawl, and the cleanup looks depressingly similar.
We Instrumented Our LLM Calls and Found the Budget Leak
A client's AI features were burning through their OpenAI budget 3x faster than projected. Adding OpenTelemetry's GenAI semantic conventions revealed the problem wasn't what anyone expected.
We Tripled Code Output. Delivery Speed Didn't Budge.
A client's engineering team adopted AI coding tools across the board. Individual velocity soared. But their DORA metrics flatlined — and the reasons had nothing to do with the tools themselves.
Your AI Coding Tool Just Got Acquired. Now What?
The SpaceX-Cursor deal forced a lot of teams to rethink their AI tooling strategy overnight. But the real problem isn't the acquisition — it's that most teams never had a strategy to begin with.