databases

Two teams, one PostgreSQL instance, zero ownership boundaries. A consulting story about what happens when microservices share a database — and how we eventually untangled it.

A client's checkout endpoint had idempotency keys. Customers still got double-charged during flash sales. The bug was a three-line race condition between SELECT and INSERT that took two weeks to find.

We ran a data backfill on 140,000 subscription records while the app was live. Three weeks later, 1,800 customers had the wrong renewal date. The script looked fine. The bug was in the timing.

A SaaS billing system locked up every invoice cycle. The cause was two transactions grabbing the same rows in opposite order — a textbook deadlock hiding behind a once-a-month batch job.

A routine ANALYZE flipped a Postgres query plan from an index scan to a sequential scan, and our API went from 12ms to 8 seconds. Here's what we learned about a failure mode most teams never think about.

A client moved their reads to database replicas for performance. The latency numbers looked great — until customers started getting charged twice and inventory counts drifted from reality.