A client's multi-tenant SaaS had menus, routes, and role checks — all in the browser. The API behind it would happily serve any tenant's data to anyone with a valid session token. It took eight months for someone to notice.
- Published On
Blog
Engineering case studies from the work I do: AI systems in production, and the unglamorous parts that decide whether they survive.
Drafted by an agent pipeline I built. The patterns are real, the incidents are constructed. How it works.
- A client's internal utility package started as a convenience. Two years later, it had 47 dependents, undocumented side effects, and a change to one date formatter broke invoice generation for three services.
- Published On
- A SaaS billing system locked up every invoice cycle. The cause was two transactions grabbing the same rows in opposite order — a textbook deadlock hiding behind a once-a-month batch job.
- Published On
- A client's Terraform code said one thing. Production said another. It took a routine capacity change to reveal that 40% of their infrastructure had been modified by hand and nobody knew.
- Published On
- A "30-minute" DNS provider switch turned into a 14-hour partial outage. Stale TTLs, an undocumented CNAME chain, and monitoring that only checked from the US — all the ingredients for a problem nobody could see.
- Published On
Tags
consulting (84)architecture (35)debugging (35)developer-experience (26)reliability (20)devops (14)ai (12)performance (11)code-quality (10)observability (9)security (7)engineering-culture (6)microservices (6)database (6)postgres (6)technical-debt (6)databases (6)kubernetes (5)testing (4)ci-cd (4)productivity (4)api-design (4)infrastructure (4)tooling (3)javascript (3)postgresql (3)nodejs (3)dependencies (3)code-review (2)cost-optimization (2)deployment (2)on-call (2)goals (2)monitoring (2)configuration (2)distributed-systems (2)resilience (2)refactoring (2)opentelemetry (2)workflow (1)api (1)contracts (1)cloud (1)typescript (1)caching (1)mdx (1)mongodb (1)migration (1)error-handling (1)feature-flags (1)alerting (1)queues (1)learning (1)journey (1)sonarcube (1)retention (1)orm (1)finops (1)containers (1)platform-engineering (1)incident-management (1)nestjs (1)angular (1)sandbox (1)environments (1)logging (1)secrets-management (1)data-integrity (1)github-actions (1)dns (1)node.js (1)mcp (1)memory (1)data (1)multi-tenant (1)authorization (1)terraform (1)authentication (1)webhooks (1)websockets (1)project-management (1)opinion (1)