Database connection pool saturated

Symptoms

db_pool_in_use sits at db_pool_size (default 10), requests queue in db_pool_waiting, and checkout logs pool timeout: no free connection after 1000 ms. Customers get 503.

Two different causes

1. Load: more concurrent checkouts than connections. In-use rises and falls with traffic, and connections come back when traffic drops.
2. Leak: some code path does not return its connection. In-use climbs steadily and never falls, even when traffic is flat. Leaks usually live on error paths: timeouts, early returns, exceptions between acquire and release.

How to tell

Plot in-use against request rate for the last 30 minutes. A staircase that only goes up is a leak. Look for warnings that happen at the same rate as the climb (for example timeouts on calls made while a connection is held).

Mitigation

• Leak: restart checkout. A restart closes every connection and buys time until the pool fills again. Raising the pool size only delays the next exhaustion. File a bug for the leaking path.
• Load: raise db_pool_size within the allowed range (rb-005).