Case study
Cutting MySQL CPU from a constant 70–80% to 25–30%
A product-wide database performance initiative: observability, query and index fixes, table redesign, and a hot-warm-cold archival strategy.
The problem
The MySQL instance ran at 70–80% CPU almost all the time. That left little headroom for traffic spikes, slowed queries across the product, and kept the database on a larger, more expensive instance than it should have needed.
Context and scale
- A primary MySQL instance serving heavy read and write traffic across the product.
- Large, growing tables shared by many features and services.
My role
I led database performance across the product. I set up the metrics and slow-query tracking, fixed many of the slowest queries myself, and ran technical sessions on how MySQL works internally so the rest of the team could optimize queries too.
Diagnosis
I monitored database metrics and slow-query logs in every environment (dev, staging, and production). That produced a ranked list of the most expensive queries and showed recurring patterns: inefficient joins, missing indexes, and queries where MySQL used the wrong index.
Solution
- Rewrote the most expensive queries, changing how joins were performed.
- Added missing indexes, reworked existing ones, and fixed queries that were using the wrong index.
- Redesigned core tables and resolved 15+ complex slow queries together with the team.
- Introduced a hot-warm-cold archival strategy for the largest tables to reduce write cost.
- Ran knowledge-sharing sessions on MySQL internals, so the improvements didn't depend on one person.
Result
CPU utilization dropped from a constant 70–80% to 25–30%. The extra headroom made it possible to downsize the database instance, saving significant infrastructure cost.
Lessons
- Observability comes first. Slow-query logs from every environment turned a vague "the database is busy" into a ranked list of fixes.
- Fix queries before buying hardware. Much of the load came from bad joins and wrong index choices, which a bigger instance would only have hidden.
- Teaching scales further than fixing. Once the team understood how MySQL picks indexes, they caught slow queries in review instead of production.
- Plan archival for large tables early. Data growth quietly turns into write cost long before anyone notices.
Technologies
- MySQL
- Query optimization
- Indexing
- Data archival
- Observability
Details are generalized to respect employer confidentiality.
Next case studyOne platform, 21 integrations: a shared auth and proxy layer for third-party APIs