Good Swimmers Drown Quietly

Lifeguards are not trained to listen for shouting. They are trained to look for the person who has gone quiet.

Drowning almost never looks like drowning. There is no splashing, no waving, no calling for help. The mouth sits at the waterline and it is busy breathing, so it cannot form words. The arms press down on the water by reflex, trying to find something solid, so they never come up to signal. From twenty metres away, a person in the last thirty seconds of their life looks like someone treading water and thinking about lunch.

The stronger the swimmer, the longer the illusion holds. A weak swimmer panics early and panics loudly. A strong swimmer holds form. Head up, breathing controlled, stroke intact, right until it is over.

I have spent most of my career on payment systems, where a bad minute turns into declined cards and people who are rightly furious. The incidents that taught me the most were never the loud ones. They were the stretches where everything stayed green for days while something underneath quietly ran out of room.

Every Resilience Pattern Is Also a Concealment Pattern

We spend years learning to build systems that cope. Retries. Circuit breakers. Connection pools. Caches. Autoscaling groups. Bounded queues. Graceful degradation. All of it is good engineering, and all of it exists to absorb pain so that the user never feels it.

Absorbing pain and hiding pain are the same action.

A retry turns one failed call into three. The caller sees success. The dependency that was already struggling now carries triple the load, and the chart that should have shown you a failure shows you a slightly busier afternoon instead.

A cache with a healthy hit ratio will happily carry a database that has become too slow to serve the traffic directly. The database is drowning. The cache is holding its head above the water. Your latency graph is flat.

An autoscaler replaces the instance that exhausted its heap before the leak has time to become an outage. The leak is still there. You are simply paying for it, hourly, forever, in instance churn that nobody reads.

A connection pool under pressure does not throw errors. It queues. That wait happens inside your own process, in a place most default dashboards never look, and it surfaces later as latency you will blame on the network.

A circuit breaker that trips and recovers twelve times in an hour is describing a system in distress, but every individual trip looks like the pattern doing its job. It is doing its job. That is precisely why you will not notice.

None of these mechanisms are mistakes. Take them out and you get outages. Leave them in and you get a system that can be in serious trouble for a long time without anything on your wall saying so.

Availability Tells You the Head Is Above Water

Uptime, error rate, median latency. These are outcome metrics. They tell you the swimmer is still at the surface. They tell you nothing at all about the kicking.

Two services can report identical availability for the same week. One idles at thirty percent utilisation and retries a handful of times an hour. The other runs at ninety four percent, retries constantly, evicts cache entries as fast as it writes them, and stays upright only because an autoscaler and a circuit breaker are compensating for each other in opposite directions. Same green square on the dashboard. Wildly different distance from the edge.

What separates them is headroom, and headroom is the least instrumented number in our industry. It is also the only one with any predictive value. Everything else is a report on the past.

This is the same trap as the planes that came back from the war. We build our confidence out of the evidence that survived long enough to be measured, and the evidence that matters is the part that never reached the dashboard.

Measure the Effort, Not Just the Outcome

Google's SRE practice names four golden signals: latency, traffic, errors, saturation. Most teams instrument the first three carefully and treat the fourth as a footnote. Brendan Gregg's USE method is blunter about it. For every resource, look at utilisation, saturation and errors, where saturation is the queue that forms once a resource has run out of room to give.

Saturation is the effort signal. It is how hard the legs are kicking. Here is what that looks like in a real system.

Not one of these will page you on its own. They are trends, and that is the whole point. Drowning is a trend that ends in an event, and we have built our entire alerting culture around the event.

Silence Is Not the Same as Health

A system with no alerts is either healthy or no longer reporting, and those two states look identical from the outside. The batch job that never started produces no errors. The metrics agent that died took its own bad news with it. The only defence is to alert on the absence of a signal you expect, which costs almost nothing and is skipped almost everywhere.

Shallow health checks belong in the same category. An endpoint returning 200 because the process is running and one HTTP thread happened to be free has told you nothing about whether the service can do its job. A health check earns its name when it exercises the path the user actually pays for. Can it reach the database. Is the pool handing out connections. Is the queue draining or only growing. Anything less is an endpoint whose sole function is to reassure you, and it will keep reassuring you right through the incident. Alerts have to describe something a person feels, which is the argument I made at length in Nobody Promised You 100% Uptime.

The Good Swimmer on Your Team

The same blindness runs straight through how we read people.

Every organisation has a strong swimmer. The engineer who never escalates, absorbs the ambiguity nobody wrote down, quietly works the weekend, and delivers on Monday. Their output looks immaculate. So nobody looks any closer.

We manage people the way we monitor systems, by outcome. Tickets closed, features shipped, dates met. Every one of those is an availability metric. Every one of them reports the head above water and stays silent about the effort underneath, which is where the load actually accumulates.

When that person finally goes under, the resignation feels sudden. It was not sudden. It was invisible, and invisible is a measurement failure, which means it was ours to fix.

Coping and Thriving Look the Same From the Shore

The lesson is not that we need louder alarms. Loud failures take care of themselves. They wake people up, they get postmortems, they get budget.

The lesson is that coping and thriving look the same from the shore. You cannot tell them apart by watching the surface, because a well built system and a well trained person both fail the same way: gracefully, competently, and without a sound, until the moment they cannot.

So change the question. Not whether the system is up, but what it is spending to stay up. Then ask the same of whoever is running it.